Posts on Seán Fobbe

[Essay] Reflections on Academic Communication

Fri, 20 Feb 2026 00:00:00 +0000

Overview Link to heading

Academia is in a crisis of communication. Academics do a lot of communicating, but we rarely reflect on how and why we communicate. There are very active debates on the reform of academic publishing, but they rarely make it into the mainstream of academia.

Both “the paper” and “the book” have become the standard units of currency in the academic world. So entrenched are they that in the modern day it is often required to publish a set number of articles in journals with a specified amount of prestige (or books with a prestigious publisher) to have a chance at an academic career. The research really is secondary, it is the venue of publication that matters to career advancement. Academics do the research, write the articles, review the articles and read the articles — for free. However, for some reason it is the commercial publishers that make fantastic profits off all this activity (see this post by Björn Brembs for more background).

I’ve been thinking about academic communication a lot and I decided to write up some thoughts as an essay, perhaps make a series out of it. By “academic communication” I mean communication as an academic, whether this is within academia among academics or if it is between academia and business, academia and civil society or academia and the general public.

In this first essay I’ve chosen to focus on some modern means of academic communication: websites, social media and e-mail. Neither of them is enough to make an academic career, but they represent some of the best means we have to evade traditional academic gatekeeping.

Table of Contents Link to heading

Website and Blogging Link to heading

I’ve experimented with and considered a lot of publishing channels and venues for my academic and personal work, but by far the one I enjoy the most is my website and blog. It is my personal corner of the internet where I can write whatever I want to, make it look like whatever I want and can try adding code, diagrams and citations wherever I please. The website is in dark mode by default, because I like it. Some people don’t and that is fine, but of all readers I probably spend the most time having to look at it, so dark mode it is.

I wouldn’t go quite so far as to recommend that every academic have their personal website, but creating websites has become so easy that I believe everyone should at least consider it. Most people think this means having a full-fledged blog and tons of content, but in reality a basic CV, a list of publications/data/code and some contact information (e-mail, social media profiles) is perfectly fine as a first website. Why? See below on gatekeeping, but the short answer is that a personal website has far fewer gatekeepers than social media or even an institutional page and is portable across corporate failures and academic transitions.

Essays Link to heading

Essays allow me to experiment with my writing, try new things, be informal, be formal, be cranky, be professional and everything in between. Some essays have citations, some have diagrams, some have code, some have none of these. My essays are certainly too long by any traditional standards of web publishing, but on my own website there isn’t really anyone who can stop me. I don’t use Large Language Models (LLM) for any of my writing, because the writing is the point of it, not the product. Personally I think this comes close to how scholarship is the most fun: chaotic, experimental, curiosity-driven and unique.

RSS Feed Link to heading

I have an RSS feed to push new posts to people using feed readers, free of charge, no account required, no tracking involved. Some people consider this ancient technology, but it is in fact a modern, important, open, standardized and decentralized means of publishing content that many ignore because we have forgotten that there are good things that are not expensive Software-as-a-Service (SaaS) “platforms” with aggressive vendor lock-in. The entire podcast industry runs on RSS and I hope that RSS will make a comeback among the general population with an easy-to-use feed reader. I use RSS a lot to get my regular news intake, check academic journals for new articles and read my favorite blogs.

Gatekeeping Link to heading

Almost every communication channel these days has some gatekeeper that must be convinced to publish your work. Some gatekeepers are more hands-off, some have very strong opinions on what can be published and how it should be written. You’d think that traditional academic publishing has the strongest gatekeepers (seeing as there is serious science to be done) but personally I think it is social media.

Social media algorithms are just brutal in their selection of which content gets attention and which doesn’t. If an academic journal rejects you, you just try the next one. It might not be as prestigious as your first choice, but there are usually at least a dozen alternatives to choose from. If your work gets published elsewhere, it’s usually just as well as long as the article itself is good, except if you really need a publication in a Nature-equivalent in your field to get a job.

With social media on the other hand, there is no easy way to switch platforms (except the Fediverse, which allows communication between all the different ActivityPub-compatible clients like Mastodon, Friendica, PeerTube, Pixelfed etc.).¹ People cross-post, of course, but most social media algorithms tend to prioritize the same type of content (catchy, short to match modern attention spans, emotional, divisive, simple), resulting in heavy pressure to conform to a particular archetype of content. If your content doesn’t fit the type, it ends up in algo-purgatory, never to be seen by more than the three or four die-hard friends who check the chronological feed on your profile every week. The most worrying development in social media is the quasi-suppression (or down-ranking) of external links, turning platforms into strictly walled gardens and cutting off means to escape the platform. This hostility to linking is really a threat to the open web and together with AI plagiarism it damages the entire information ecosystem.

By contrast there is no real gatekeeper to your website’s content. Yes, discovery can be an issue if Google doesn’t send generic traffic your way. With “AI search” this is going to become much worse, because the AI algo-washes your content and presents it as its own — without attribution, of course.² However, there are many other ways of discovery, some which can be just as relevant to academics as Google. Academics don’t need millions of visits per day, they need a dozen relevant people in their field to deeply engage with their ideas.

People learn about you when you meet them for coffee, when you meet them in person, when you give a presentation at a conference, every once in a while through social media posts, through personal recommendations and so on. As soon as people hear about you as an individual and understand that you have a website, you have potentially broken Google’s hold and freed yourself from all gatekeepers forever. Usually this takes years and sustained work, but it is an option.

Asynchronous Publishing Link to heading

A website stores and presents everything you put on it, whenever you choose, in whatever structure and design you choose. You might merrily go your way publishing for months or years before a potential reader learns about you, but when they do, they have access to all the content you published before. This gives them a ton of material to choose from, to establish your legitimacy as an author and to forward to other people, if they think it’s good. One of my happiest internet experiences is discovering a mature blog I had overlooked before and going through the archives to see what else the person has written. I enjoy forwarding links to old articles, because some ideas are still valuable years later.

The contrast is, of course, social media feeds, where anything you write pretty much vanishes after a day or two, never to be seen again, except be intrepid explorers of your profile who can stomach scrolling through your unstructured flow of thoughts at length on a platform that makes it as painful as possible. I wonder how much wonderful ideas we lose forever, simply because social was the only place they were ever posted and they never received the audience they should have because the computer said no.

Speed Link to heading

This may be a pet gloat of mine, but my website loads fast. It’s not because I added some clever technology, but because of the absence of technology. There is (almost) no Javascript that must be loaded to display the website,³ there are no tracking scripts that must be loaded, there is no overloaded Javascript framework involved, just some HTML, CSS and occasionally some down-scaled image files, if the page or post warrants it. Contrast this with bloated sites like GitHub or Linkedin that take several seconds to load, even on a 100 MBit connection and a modern computer. Even regular blogs these days are so full of unnecessary frameworks that it makes me wonder why I am loading all this Javascript to read some simple articles. And I do have to load it explicitly, because I have NoScript on by default. Since I have to check and re-check my website often to revise it and upload new content, its speed makes this a surprisingly enjoyable experience.

Analytics Link to heading

My website doesn’t have analytics. While I have some strong philosophical opinions on tracking and generally oppose it, the best part about being tracking-free is that I save a ton of time not looking at analytics. Tracking never mattered to the purpose of my website in the first place. Checking analytics all the time can consume you.

In the past I spent much too much time looking at analytics on Linkedin. What did I learn from it? Nothing useful, really. Some posts do really well, some don’t. I often can’t say why, except it being down to luck that a few people liked the post in some crucial early window during which the algo decided whether to promote it and then it spread fast. A lot of people distribute likes primarily to those whom they like and respect as a person to help their content get noticed. This is great, but again, I can’t tell if people like the content or not unless they leave a substantive comment. Thoughtful comments are rare and again, you’ll usually get them from people who enjoy your work in general, not so much from a cross-section of the population who honestly respond to your work.

In the end, analytics do not measure what matters to me. It matters to me that people closely engage with my writing, learn from it, perhaps change their behavior and become better legal and technical professionals. I usually discover if this worked when people write me thoughtful e-mails, have a good conversation at a conference or recommend my work to other people. None of these feedback signals can be captured by a regular analytics platform, so I don’t bother. In the years I’ve been running this website, I never felt like I made a mistake.

Social media is kind of the new kid on the block in terms of digital distribution channels, even though it has been around for years now. Social media quickly become critically important for academics to disseminate their research, in a way that it is hard not to have a social media presence as a young academic these days. I’m not sure if Facebook was ever important for academics, but Twitter quickly became the go-to place for “serious” social media accounts, including academics, journalists and politicians. Of course Twitter later fell off a cliff after the Musk takeover and turned into a Nazi hellhole now called X, but it was useful and valuable to academics while it lasted.

In most disciplines in the academic world there is a sliding scale of importance in academic journals. Papers published in “good” journals traditionally receive more attention than papers in “mediocre journals”. Journal importance is both blessing and curse. On the one hand, getting into the right journal provides much-needed attention for good work, but on the other hand, the more important the journal, the more conservative it is with regards to methods and content. Social media levels the playing field a little. Even if you don’t make it into Nature, social media is still a chance for your research to be noticed. Perhaps only be your peers in your circle of PhD students, but you’ve got to start somewhere. This is especially important for people like me who work with very much non-traditional methods in a very conservative discipline (law). Social media often allows me to make an end-run around the traditional gatekeepers.

Mastodon Link to heading

Mastodon is my absolute social media favorite. I created my account in November 2022 during the big Twitter migration, have been active on the site for three and a half years and haven’t regretted it at any point. Mastodon does a lot of things right in a way similar to early commercial platforms, before they deteriorated in quality to wring out every last bit of attention from their users and advertisers.⁴

These are my personal Mastodon highlights:

Unexpected and unique perspectives that would be canned by any standard social media algorithm
Plenty of academics from all disciplines, expert tech people and non-traditional individuals
Mastodon software is open source
Powered by the ActivityPub protocol, an open standard for social networking
Developed by a non-profit
No suspicious algorithm
No ads in the software itself (however, some accounts, esp. companies, post ads as part of their regular output, so Mastodon is technically not ad-free)
Limited analytics (counts of boosts/favorites/followers, but that’s it)

While there are many good sides to Mastodon, it admittedly is a relatively small network compared to the major social media behemoths. The Twitter migration brought a massive influx of new users, but many didn’t stay with Mastodon. The website mastodon-analytics.com indicates that Mastodon had around 400,000 monthly active users (MAU) in mid-2022, peaking at around 2.5 million MAU after the Twitter meltdown later in 2022. The Fediverse Observer shows we are currently at around 750,000 MAU in February 2026. So Mastodon almost doubled its long-term active user base over a few years. Total monthly users are around 9.5 million, which is small compared to Linkedin’s claimed 1 billion users. However, there is still the wider Fediverse which is interoperable with Mastodon, and that adds a good number of users.

In the long-term I am confident that Mastodon will win in social media market share. Either Mastodon itself or other open social media platforms will slowly gain dominance. This may take decades, but I’m in this for the long run. UI/UX is improving slowly but consistently. A lot of people come to stay. However, the most important feature is its technical foundation on open standards and open source. Linux took a long time to take off, but it has gained in reputation and legitimacy every decade and is now threatening even traditional Windows strongholds such as gaming and government use.

Linkedin Link to heading

Yes, I have a Linkedin account and I maintain it semi-actively. I am not terribly excited about Linkedin (nor Microsoft) and have never been, but I must admit Linkedin has been useful at times. The German legal community is quite active on Linkedin, same as the business community in general. I believe this is deteriorating due to AI overload, but more on that later.

Working conditions in academia are quite poor in Germany (low pay, mostly part-time and short-term contracts, very few permanent positions, strict hierarchies), so it is helpful for academics at all stages pre-tenure to be able to earn money in industry. I’ve done so for a number of years now and the mix between low-paid/unpaid academic work and well-paid industry work has allowed me to succeed in paying the bills and keeping me intellectually curious.

In terms of maintaining one’s sanity it would certainly be better to read fewer posts from salespeople (including most CEOs, by the way) trying to sell inferior products with outrageous lies about productivity gains to scared customers traumatized by the AI hype machine. Unfortunately, not everything in life pleasant. Sometimes we must endure. I certainly have won jobs, contracts, connections and a reputation from being active on Linkedin. For, now, I think trading some of my sanity to be able to feed my family is worth it, but I do wonder if this will be the case for much longer. I find the value of Linkedin has been steadily decreasing since the release of ChatGPT.

Despite the huge difference in the size of the user base between Linkedin and Mastodon, I see more high-value posting activity on my Mastodon feed than on Linkedin. I follow about 3,000 to 4,000 people on each site, so I should roughly be seeing a similar amount of useful content on each site. However, since the advent of AI the number of valuable posts from people I find interesting has dropped off a cliff on Linkedin and hasn’t recovered. I assume it is some combination of the algorithm having become bad at detecting human posts, AI slop completely overwhelming the system and people deciding to opt out of posting because of poor reach. I see this at work in my own posting habits. The reach of my Linkedin posts and reactions to them are down to about 10% of what they were before 2025. At some point you start to wonder whether the effort is worth it.

Linkedin knows this. Some time in 2025 they changed the creator dashboard to not show daily impressions/reactions by default, but instead only show the cumulative impressions/reactions per interval (7 days by default). What this means, of course, is that the numbers and the line always point upwards. You don’t do this if you are confident that many active people read and comment on timely and valuable posts. This change, more than anything else, tells me that Linkedin is failing, possibly irreversibly.

My personal approach to Linkedin at the moment is to avoid reading the feed, check for messages and comments on published posts every couple of days and occasionally post something about my work. Since 2025 I have rarely written posts targeted at a Linkedin audience, but instead write them for Mastodon and cross-post them if it feels right. I still announce larger items (papers, open source projects, blog posts) on Linkedin, but again with the text that I write for Mastodon.

E-Mail Link to heading

E-Mail is old. It primarily used to be a means of direct communication among individual scholars and groups of scholars. There were newsletters, of course, but this usually indicated some kind of institutional setting, making it necessary to regularly communicate information top-down and one-to-many.

This has changed in recent years with the rise of professional newsletter writers who produce a regular newsletter for a group of paying subscribers and, as far as I can tell, for some academics and journalists this is a viable means of making an income or at least some nice side-money. This is has been made easier by SaaS platforms targeting this market segment, which I prefer not to name because you probably have heard of them and don’t hold them in high regard. Nevertheless, technological developments have made publishing professional paid newsletters much easier than before.

While e-mail is still my medium of choice for professional communication among peers, I’ve opted against publishing a regular newsletter or even implementing some kind of RSS-to-E-Mail mechanism that mails out updates from my website to subscribers.

There are a number of reasons why I do this:

E-mail channels are overloaded. Most people I know are drowning in e-mails. No one knows what inbox zero looks like. Add to that the fact that very few people know how basic keyword-style e-mail filtering works, inboxes around the world are exploding with e-mail. Why write a newsletters that just gets lost in a mountain of text?⁵
There are too many newsletters. There are all the traditional newsletters related to work and civic life one has to watch. There are many more new newsletters being created to look for a viable source of income as a writer. There are all the unwanted newsletters people add me to whether I like it or not because I needed an account somewhere and some e-commerice business development executive thinks that is an invitation to bombard me with e-mails every week. I have no idea how people who deal with this without e-mail filtering.
Newsletter writing feel like a chore. I enjoy writing when the urge hits me and I don’t enjoy writing when it doesn’t. Turning occasional blogging into a regular newsletter would require me to publish new articles on a regular basis to not lose the attention of my audience and I just don’t feel like turning something fun and intellectuall freeing into a chore.
Managing a subscriber database is work. Building an e-mail subscriber base requires time, persistence and a lot of marketing. Then you have to make sure the e-mail database stays safe and never leaks. All in all even with technological assistance it is a lot of work that I don’t need to do if I stick to blogging.
Long-term storage of witing. A blog keeps everything as long as I want it to. People delete e-mails. People should delete e-mails, particularly newsletters, to free up space and retain their sanity. My own e-mail program is set to auto-delete e-mails from certain folders and my own newsletter folder is set to auto-delete everything older than 30 days. On my blog I can keep my articles around for as long as I like, in whichever order I like.
Editing and Updates. Sometimes I make mistakes. Sometimes I want to update some post with new information. When you hit “send” on an e-mail it is gone and issuing corrections or updates requires sending new e-mails, irritating everyone. Blogging can do that, e-mail cannot.

There are certainly valid reasons for pursuing a newsletter writing strategy as an academic or journalist. Human attention from search is becoming less and so-called AI produces a ton of traffic only to plagiarize content and keep all the valuable human attention for itself. Controlling distribution via an individual subscriber database sounds nice, but in practice this is only viable for the most popular writers. All others end in social media recommendation hell on one of the typical newsletter platforms, trying to please the recommendation algorithm while never building a subscriber base large enough to leave said platform.

I’m fine with blogging.

There are issues with portability in the Fediverse, too and moving posts in particular seems difficult, but it does a lot better than most. ↩︎
This is a human em-dash, by the way. One of the least enjoyable features of this AI timeline is that using em-dashes makes you seem like a robot. ↩︎
I do have a dark-to-light-mode switcher for people who need it for accessibility reasons and some tutorials contain some minor Javascript to display visuals, but that’s it. ↩︎
Twitter, of course, did not just deteriorate but went full Nazi algo with a built-in bot that could commit sex crimes 24/7. I have very low moral expectations of billionaires, but even I didn’t expect this level of moral decay. ↩︎
E-mail filtering can perform certain actions on e-mail triggered by rules. Think for example redirecting all tech newsletters to a specific folder in your mail account and redirecting all shopping-related things (order confirmation, invoice, delivery status) to another folder for quick browsing. With known senders it is often enough to simply use the sender address as a hook and this works well for all types of newsletters. ↩︎

200,000 Legal Dataset Downloads

Mon, 17 Nov 2025 00:00:00 +0000

Info

Open Data Repository — data.seanfobbe.com

Thank you! Link to heading

Thank you all for over 200,000 legal dataset downloads!

In the past few years I have published 19 open, ethically sourced and reproducible legal and political datasets on a wide variety of institutions and subjects, among others:

International Court of Justice (ICJ)
Permanent Court of International Justice (PCIJ)
UN Security Council (UNSC)
International treaties
German federal courts (BVerfG, BGH, BVerwG, BFH, BPatG, BAG)
German federal law
German federal parliamentary debates and materials
Auxiliary datasets (e.g. stop words, docket numbers)

You can download all of these datasets Open Access via the open data tab on my website.

Zenodo Download Statistics Link to heading

Applications Link to heading

If you happen to have built and published any interesting applications or research with these datasets, write me and let me know!

[12 September 2025] Upcoming Agora Talk at the European Society of International Law (ESIL) Annual Conference in Berlin

Mon, 07 Jul 2025 00:00:00 +0000

Upcoming Talk on New Methods in International Law Link to heading

I’m excited to be speaking at the 2025 European Society of International Law (ESIL) Annual Conference! The conference is being held in Berlin (Germany) from 11 to 13 September 2025 under the general theme of “Reconstructing International Law”.

My talk is titled “The Chains of Pride: Reconstructing International Law as a Discipline of Fact and Principle” and will be part of Agora 8 on “Interdisciplinarity and the (Re)construction of International Law”.

The key message I want to get across is that we urgently need to enhance international law with new methods. This is not a call for interdisciplinarity, but for changing how the core of the discipline is taught and practiced under a serious reading of the Vienna Convention on the Law of Treaties (VCLT). For example, the ordinary meaning is an empirical question of how language is used in practice and the interpretation of the object and purpose of a treaty pretends to be a normative exercise, but in truth it is a powerful demand to align select empirical causes with intended practical effects.

See the abstract below for more details.

Timetable Link to heading

Panel	Date	Time	Title
Agora 8	12 September	14:00	The Chains of Pride: Reconstructing International Law as a Discipline of Fact and Principle

Agora 8 will feature other great talks on social science and international law:

Reconstructing International Treaty Law-Making with Behavioral Research (Professor Anne van Aaken, University of Hamburg, and Professor Tomer Broude, Hebrew University of Jerusalem)
Insights from Interdisciplinary Research on Human Rights Treaties at the Subnational Level (Professor Evelyne Schmid, Université de Lausanne)

Be sure to check out the conference website and conference program for more information.

The Chains of Pride (Abstract) Link to heading

The autonomy of international law has always been a mirage. Power, empirical reality and fait accompli on the ground shape international law like no other legal field. Empirical facts are embedded in the very foundations of the system: from the effective control that makes or breaks a government to the political practice of States that creates international custom and influences the interpretation of treaties. International law may be the structured and principled application of power, but it is an application of power.

Even the traditional tools of the international lawyer are empirical at heart. The ordinary meaning in Article 31 (1) VCLT is an empirical question of how language is used in practice. The interpretation of the object and purpose of a treaty pretends to be a normative exercise, but in truth it is a powerful demand to align select empirical causes with intended practical effects.

The international legal discourse has done the world a disservice by discouraging the teaching of empirical methods and by insisting that international law is primarily a normative discipline. The rational creation and application of the law requires skill with both normative and empirical methods. In its pride, international legal scholarship has limited itself unnecessarily. Now is the time to rebuild the discipline.

I will show how three new datasets on the ICJ, PCIJ and UNSC help us push the boundaries of legal analysis. Natural language processing permits the symbiosis of close and distant reading (Moretti 2000). Statistical techniques help us see the forest among trees in international relations. Causal inference provides several convincing frameworks to rationally analyze and debate the practical impact of legal norms. Network analysis reveals not just citation habits, but a whole new world of relations in society.

We have nothing to lose but the chains of our pride.

Slides for 2025 ESELS Toulouse Conference Talks

Wed, 25 Jun 2025 00:00:00 +0000

Info

Slides for 2025 ESELS Conference Link to heading

The slides for my conference presentations from the 2025 European Society for Empirical Legal Studies (ESELS) Annual Conference in Toulouse are now online!

In case you missed the talks, you can now get the slides from Zenodo. Click the links in the box above to go straight to the PDF files.

The abstracts for both presentations are available on my website.

See you next year in Ljubljana!

[19-20 June] Upcoming Talks at the European Society for Empirical Legal Studies (ESELS) Conference in Toulouse

Sun, 25 May 2025 00:00:00 +0000

Info

[Open Access] “Words of Power: Introducing a Comprehensive Corpus of UN Security Council Resolutions”
[Open Access] Corpus der Entscheidungen des Bundesgerichtshofs (CE-BGH)
[Open Access] Entscheidungen des Bundesgerichtshofs in Strafsachen aus dem 20. Jahrhundert (BGH-Strafsachen-20Jhd)

Upcoming Talks at European Society for Empirical Legal Studies (ESELS) Annual Conference Link to heading

More Empirical Legal Studies talks coming up! I’ll be speaking at the European Society for Empirical Legal Studies (ESELS) Annual Conference (19–20 June 2025) in Toulouse — twice.

On 19 June I will discuss efforts (with Tilko Swalve) to produce corpora of all judgments and decisions rendered by the German Federal Court of Justice (BGH). I’m especially excited about the UNSC talk on 20 June, where I present our new pre-print entitled “Words of Power: Introducing a Comprehensive Corpus of UN Security Council Resolutions”, co-authored with Lorenzo Gasbarri and Niccolò Ridi, already available open access.

Be sure to check out the conference website and conference book for other great talks!

Schedule Link to heading

Panel	Date	Time	Title of Presentation
2.2	19 June	14:00 – 15:30	Introducing Corpora of Judgments for the German Federal Court of Justice (Bundesgerichtshof) in Criminal Matters (1950-1999, 2020–2024) and Civil Matters (2000-2024)
6.1	20 June	16:00 – 17:30	Words of Power: Introducing a Comprehensive Corpus of UN Security Council Resolutions

UN Security Council (UNSC) Talk Link to heading

Words of Power: Introducing a Comprehensive Corpus of UN Security Council Resolutions

The United Nations Security Council is the most influential and powerful of the principal UN organs. Article 24 of the UN Charter assigns the UNSC “primary responsibility for the maintenance of international peace and security”. Chapter VII of the Charter grants the Council unique binding powers, but the UNSC plays a prominent role in all affairs of the UN.

We introduce a novel international legal dataset containing texts and metadata for all resolutions of the United Nations Security Council (UNSC) from resolution 1 (1946) through resolution 2722 (2024). The dataset provides resolution texts in all six official UN languages, draft texts and meeting records in English, as well as extensive metadata, for a grand total of 82 variables.

The computational workflow is engineered end-to-end as a fully automated extract-transform-load (ETL) data pipeline with citation analysis and NLP components, unit tests and complex reporting. The declarative workflow is fault- tolerant, resumable and stores intermediate results in over one hundred individual checkpoints. The underlying code is published open source under the GPL-3.

In an international community founded on the rule of law the activities of the United Nations must be public, transparent and defensible. In these troubled times where the rule-based international order is under attack it is more important than ever to document, understand and promote systematic approaches to international law.

Keywords: United Nations, UN Security Council, Data Engineering, Network Analysis, Natural Language Processing

Federal Court of Justice (BGH) Talk Link to heading

Introducing Corpora of Judgments for the German Federal Court of Justice (Bundesgerichtshof) in Criminal Matters (1950-1999, 2020–2024) and Civil Matters (2000-2024)

The Bundesgerichtshof (BGH), the German Federal Court of Justice, is the highest court of appeal in civil and criminal matters in Germany. BGH judgments receive regular and intense scrutiny, but German doctrinal scholars routinely focus only on a single decision or a small set of decisions. Quantitative approaches are often infeasible because of lack of data.

We present two original datasets on the German Federal Court of Justice: 1) a corpus of all 77,892 judgments in civil and criminal matters for the period 2000-2024 with 36 variables plus the internal citation network and 2) a corpus of 36,316 judgments in criminal matters for the period 1950-1999 with 31 variables.

We use state-of-the-art data engineering techniques to construct reproducible extract-transform-load (ETL) pipelines that automatically acquire, clean, test, analyze and document German Federal Court of Justice data from the official BGH database and an older criminal justice dataset provided to the authors. Citations are extracted from judgment full texts, analyzed and provided in GraphML format.

Despite centuries of intense academic engagement with the law, quantitative and empirical approaches remain exceedingly rare in Germany. Lack of data, lack of training, lack of awareness and many other reasons have caused this situation. Solving the data problem is a first step to empowering German lawyers in the use of quantitative methods and may help with an empirical turn.

Keywords: Bundesgerichtshof, Criminal Law, Civil Law, Data Engineering, Network Analysis

[28 May 2025] Upcoming Talk on Legal Data Engineering (Online)

Tue, 13 May 2025 00:00:00 +0000

I’ll be speaking about Legal Data Engineering at the Legal Tech Lab Cologne on 28 May 2025!

We’ll discuss the foundations of Legal Data Engineering (as a sub-discipline of Legal Data Science), the practice of Legal Data Engineering and its legal framework in Germany.

There will be opportunities for Q&A and networking, I’m told.

When? 28 May 2025 at 7 pm (CEST)
Zoom link: https://us06web.zoom.us/j/85612568748
Password: 691761

Impatient? Felipe Molina and I spoke about Legal Data Science on the Talking Legal Tech podcast and the recording is available online.

[New Pre-Print] Words of Power: Introducing a Comprehensive Corpus of UN Security Council Resolutions

Fri, 11 Apr 2025 00:00:00 +0000

Info

[Open Access] Words of Power: Introducing a Comprehensive Corpus of UN Security Council Resolutions (Pre-Print 2025)
[Open Access] Corpus of Resolutions: UN Security Council (CR-UNSC)

New Pre-Print on UN Security Council Data Link to heading

New Pre-Print! We introduce the Corpus of Resolutions: UN Security Council (CR-UNSC), a novel international legal-political dataset containing texts and metadata for all resolutions of the United Nations Security Council (UNSC) from resolution 1 (1946) through resolution 2722 (2024), as published by the UN Digital Library (UNDL).

The dataset provides full resolution texts in all six official UN languages, draft texts and full meeting records in English, as well as dozens of metadata variables, for a grand total of 82 variables in structured tabular CSV format, accompanied by extensive narrative documentation. In addition to the main tabular dataset the CR-UNSC is accompanied by specialized variants for network analysis and bibliography management.

The network analysis variant of the dataset offers citation network data in GraphML format containing all full-text citations to UN Security Council and to UN General Assembly resolutions associated with 54 node-level metadata variables and citation counts.

The bibliography management variant reproduces a significant part of the dataset metadata in BibTeX format ready for import into widely used reference managers such as Zotero, Jabref, Citavi and Endnote. The bibliographic database ensures inclusive access to the dataset for traditional researchers with no expertise in data analysis.

The computational workflow is engineered as a fully automated end-to-end extract-transform-load (ETL) data pipeline with citation analysis and NLP components, unit tests and extensive reporting. The declarative workflow is fault-tolerant, resumable and stores intermediate results in over 100 individual checkpoints. The code is published open source under the GNU General Public License Version 3 (GPLv3).

We intend to update the corpus at least once per year. The most recent version of the CR-UNSC corpus will always be available open access via Zenodo through its Concept DOI located at doi.org/10.5281/zenodo.7319783.

Author Websites Link to heading

Seán Fobbe — https://www.seanfobbe.com

Lorenzo Gasbarri — https://www.lorenzogasbarri.com/

Niccolò Ridi — https://www.kcl.ac.uk/people/niccolo-ridi

Migrating my Open Source Projects to Codeberg

Thu, 10 Apr 2025 00:00:00 +0000

Open Source Development now on Codeberg Link to heading

Effectively immediately, I have migrated all of my Open Source projects from GitHub to Codeberg.

Future Open Source development will continue on Codeberg only.

All GitHub repositories will remain available as public archives to ensure that nothing breaks for other people and the scholarly record is maintained. However, please look to my Zenodo code archive if you are looking for stable and citeable code releases.

Why Codeberg? Link to heading

I’ve become increasingly concerned about the future of open source projects hosted by commercial companies, more specifically companies based in the United States. The wholesale turn towards AI in the enterprise world and its pillage of the open software and open culture ecosystem has been quite bad already. However, add to that the erratic, immoral and brutal behavior of the second Trump administration, which is likely to grow worse over time. It is quite probable that every bit of US soft power will be turned into a political weapon before long, including the market dominance of its tech companies.

What could this mean in practice? A shutdown of “leftist” open source projects? Account bans for human rights lawyers? Sharing of personal data on “undesirables” with law enforcement? Intentional censorship of source code? I have no idea, but the tariff saga, enforced disappearance of persons in the US and the constant attacks on the rule of law should make anyone wary of relying on US infrastructure.

Therefore I’ve chosen to migrate all of my Open Source projects to Codeberg, a git hosting platform run by the eponymous German non-profit organization. Codeberg is organized as a collective membership-based non-profit and I’ve additionally chosen to join the non-profit as a member because I believe it is important to keep the stewardship of open source infastructure in the community instead of outsourcing it to commercial companies.

Technically speaking, Codeberg has impressed me.

The Codeberg web interface is much (!) faster and more responsive compared to GitHub
The Git clone and push feel about twice as fast as GitHub (not that it matters much)
Much cleaner interface than GitHub, esp. less bloat and less featuritis
No silly AI tools being pushed into my face all the time
Codeberg activity overview is very helpful and well organized
Codeberg migration tool works well
The OSS git platform software that Codeberg runs is called Forgejo, a fork of Gitea.

I look forward to exploring Codeberg and engaging with the non-profit organization in the future!

Notes on the Migration Link to heading

The migration from GitHub to Codeberg was largely painless, although moving, checking and archiving ~30 repositories individually took some time.

Codeberg offers an excellent migration tool that can copy repositories from GitHub and other Git hosting platforms, including wikis, LFS files, issues, pull requests, labels, milestones and releases. I primarily migrated issues and releases, but everything looks like it is in order.

Codeberg does offer private repositories, but limits them to 100 MB in size because the organization wants to promote open source, not private development. I believe this is fair.

The Censorship of Science in the United States

Fri, 04 Apr 2025 00:00:00 +0000

I’ll be honest, I don’t intend to submit academic papers to US-based journals and conferences anymore.

Self-censorship of science at US universities and scientific organizations is in full swing and I see virtually no institutional resistance (although some individual scholars are protesting). I don’t expect to get a fair hearing with human rights-oriented empirical legal research in peer review any longer.
The way this is going I would expect even currently neutral journals to fold within the usual timeframe of a paper review process. I don’t feel like burning a paper in the hopes of getting it published before the door closes.
I prefer to submit my research to journals backed by institutions that support human dignity, diversity and the freedom of scientific research.

We’ve all seen where this went in Europe during the 1930s. I hope the US will learn from our history, but I am not optimistic right now.

[Essay] The Intelligent AI Coin: A Thought Experiment

Fri, 21 Feb 2025 00:00:00 +0000

Overview Link to heading

Recent years have seen a concerning trend towards normalizing decisionmaking by Large Language Models (LLM), including in the adoption of legislation, the writing of judicial opinions and the routine administration of the rule of law. AI agents acting on behalf of human principals are supposed to lead us into a new age of productivity and convenience. The eloquence of AI-generated text and the narrative of super-human intelligence invite us to trust these systems more than we have trusted any human or algorithm ever before.

Frank Herbert captured the essence of this looming danger in Dune (Herbert [1965] 1968, 23):

Once men turned their thinking over to machines in the hope that this would set them free. But that only permitted other men with machines to enslave them.

It is difficult to know whether a machine is actually intelligent because of problems with construct validity, plagiarism, reproducibility and transferability in AI benchmarks. Most people will either have to personally evaluate the usefulness of AI tools against the benchmark of their own lived experience or be forced to trust an expert.

To explain this conundrum I propose the Intelligent AI Coin Thought Experiment and discuss four objections: the restriction of agents to low-value decisions, making AI decisionmakers open source, adding a human-in-the-loop and the general limits of trust in human agents.

Table of Contents Link to heading

The Rise of LLM-based Decisionmaking and Agentic AI Link to heading

The rise of Large Language Models (LLM) has been accompanied by a concerning trend: more and more people are expressing an interest in allowing LLMs (or, more often, a service built on an LLM, e.g. ChatGPT) to make decisions for them. On the public internet this often takes the form of grandstanding social media posts that begin with “I asked ChatGPT and this is what it told me to do!”.¹

This trend is related to the theme of agentic artificial intelligence (“agentic AI”), which is usually taken to mean some form of captive, docile and benevolent AI that takes general instructions, performs tasks and makes minor decisions on behalf of a human principal. Sort of like having a servile fantasy employee willing to go a dozen extra miles — just without the irritating need to pay a human being a living wage. We’ve seen this trope many, many times in science fiction. Justified or not, a large number of people believe that the time for science fiction to become reality is now.

The reason I am concerned about LLM-based decisionmaking and agentic AI is not because someone might buy the wrong pet food or that people could be ripped off when the AI servant falls for some a scam on Amazon Marketplace. This type of problem may become a serious consumer protection issue if agentic AI takes off in the mainstream, but it will remain manageable. The true danger lies in the damage done to entire social systems, particularly through democratic backsliding and an erosion of the rule of law.

Unfortunately, LLM-based decisionmaking is already being trialed for the creation of laws, the writing of judicial opinions and a lot of routine administrative work that bores people to tears but is in fact fundamental to the functioning of the rule of law. The trend towards AI decisionmaking normalizes the delegation of complex decisions to opaque systems. Systems that are ripe targets for capture by hostile interests and actors who prefer dictatorship over democracy and promote nepotism over the rule of law.

Freedom through Machine Intelligence? Link to heading

Of course, science fiction was there first. Frank Herbert explained the problem in Dune (Herbert [1965] 1968, 23):

Once men turned their thinking over to machines in the hope that this would set them free. But that only permitted other men with machines to enslave them.

Consider Twitter, now known as “X”. Twitter began as a passable attempt at creating a digital town square. Now, in 2025, Twitter/X has lost most of its value as a public space and has been transformed into a bullying tool for an out-of-control billionaire hell-bent on riding the rising tide of fascism, destroying his enemies and enriching himself even further. The algorithm now serves the purposes of one man, at the expense of many.

Imagine what might happen if we delegate not just control of information flows, but entire decisionmaking chains to black box systems that can be captured by anyone with a few billions and a vendetta. Perhaps these decisionmaking systems will even be purpose-built to exert covert influence, presenting a more benign image than X to the world. You wouldn’t even need to bribe all those elected representatives with generous campaign donations if you can control the computer system that tells them what the “best” course of action is.

This has obviously happened before to social and technical infrastructure of public importance. History knows of more than just a few outrageous corruption scandals, regulatory captures and democracies damaged beyond repair. Nor is predatory algorithmic decisionmaking an entirely new phenomenon. We’ve seen flash crashes caused by nefarious trading algorithms, automated sentencing decisions dripping with hidden racism and lives destroyed by false allegations of welfare fraud based on discriminatory algorithms.

Then again, corruption and democratic backsliding in the age of LLMs are likely to occur on an entirely different level of opacity and scale. This is because the eloquence of AI-generated text and the narrative of super-human intelligence invite us to trust these systems more than we have trusted any human or algorithm ever before.

And I am desperately afraid that people will extend that trust.

How Would You Even Know If Your AI Is Intelligent? Link to heading

There has always been intense debate about what “intelligence” means and whether our newest computer algorithms have achieved some form of intelligence roughly comparable to modern living creatures. Over the course of the past few years alone LLMs and their derivative services have been compared to everything from cats and dogs, to toddlers and to “infinite interns”.² Recent claims have even floated “PhD-level” intelligence for systems that take time to “think” about their answer.

And of course there’s always that type of influencer who proclaims the advent of super-intelligence with each major product release.

Bless their little influencer hearts.

Note

On Super-Human Intelligence

I am skeptical of “super-human intelligence” claims made by AI evangelists. Specialized tools like machines and computer programs routinely solve certain classes of problems with super-human speed and quality — that is the entire point of making specialized tools.

Your trusty hand-held calculator can do arithmetic (and usually other kinds of math) with a speed and accuracy that cannot be matched by any human being on earth. However, framing this ability in terms of “intelligence” isn’t really helpful in understanding how a calculator works.

The definition of intelligence and our ability to artificially create it are complicated problems, but I’d like to focus on a simpler question:

How would you even know that your AI is intelligent?

I mean you, the regular person without an advanced degree in an AI-related field and no expertise in building or benchmarking models.

With humans this is sort-of easy. In the modern age we assume that all humans have a problem-solving mental ability we call “intelligence”, an ability that differs only in degree, not in kind. If a life-form is human, it probably has its own mind and possesses and can use intelligence by default, barring any serious physical or mental conditions. It’s not that we can easily tell if people really are intelligent or just clever automata.³ We assume intelligence based on their humanity. Historically this wasn’t an easy development and racism got in the way rather often, but we got there eventually. Intelligent is often understood to mean “thinks and solves problems like a human being”.

Where this assumption breaks down is when we try to apply the concept of “intelligence” to non-humans and machines.

Scientists have tried to define intelligence, break it down into individual components and to test humans and machines against these expectations to measure their true intellectual ability. Human IQ tests contain a range of verbal, numerical and spatial reasoning tasks. The raw test results for a test-taker are benchmarked against the performance of other humans. AI benchmarks consist of a similar range of reasoning tasks against which machines are tested and then compared to other machines (and humans, in some cases).

Human IQ tests do not measure the full spectrum of human mental ability, as they mainly focus on abstract tasks, not on social and emotional capacity. This is why their construct validity — meaning whether they adequately measure the concept of intelligence — has been called into question.

AI benchmarks face similar problems and some of their own (Reuel et al. 2024):

Construct validity is as questionable with AI benchmarks as it is with human IQ tests. What is intelligence? What is reasoning? If we fail to define either for humans, how can we define it for AI? And if we successfully define them for humans, does this tell us something about machines?
Cheating is a challenge because both the questions and answers for a test are available online somewhere. Often it is not clear if the tested system actually performed some kind of reasoning or simply plagiarized the answer from its considerable corpus of training data. Cheating invalidates human test performance and so it should invalidate AI test performance.
Reproducibility is a problem related to cheating: if the test and its answers are published, new AI systems will simply ingest the test and plagiarize the answer — reminiscent of a human test-taker bringing their notes to a closed-book exam. However, if the test and the answers are not published, it is very difficult to independently reproduce the test and evaluate the quality of the benchmark.
Transferability means the gap between test and practical performance. If a human performs well on an artificial IQ test, will the human succeed against real problems? If the LLM performs well on an artificial benchmark, will it be able to perform useful tasks in the real world?

Personally, I don’t think AI benchmarks are helpful in deciding whether a particular (version of an) LLM is useful, high-quality or not. Most people will not be using a benchmarked, version-controlled model directly, but some kind of service that adds layers and layers of complexity on top of the actual model to modify the user experience. These services may or may not be comparable to the benchmarked model and, even if the services are benchmarked directly, they will usually change often and unpredictably, invalidating the benchmark results.

The Intelligent AI Coin Thought Experiment Link to heading

Most people aren’t scientists and do not understand, follow or care about AI benchmarks. If ordinary people want to rely on an AI tool they must do one of the following:

Personally evaluate the quality of an AI tool against the benchmark of their lived experience, or
Trust in the word of experts that an AI tool is actually “intelligent” according to some expert definition

To give you a feel for the problem I’d like to propose a thought experiment: the Intelligent AI Coin.

Let as assume you are willing to delegate your decision problems to an external agent. For simplicity’s sake, let us assume that these are classical decision problems with only two answers: “yes” and “no”.

You come to me for help in selecting a tool to solve your problems. I offer you two identical-looking coins that can make yes/no decisions for you:

A traditional fair coin
A super-intelligent AI-powered coin

The Traditional Fair Coin is one of those legendary objects found in every statistics textbook. “Fair” means that if you flip the coin an infinite number of times, then in the long run it will show heads 50% of the time and tails 50% of the time. Using a fair coin will quickly get you a “yes” or “no” answer to your decision problem. The result will be completely random and take into account zero information from your actual problem. But you do get a result and it’s a time-honored method of deciding if you can’t do it yourself.

The Intelligent AI Coin is powered by a state-of-the-art advanced neural network decision making algorithm that can understand your problem with human compassion, reason about it with PhD-level intelligence, possesses a perfect moral compass, can access the entirety of human knowledge, is friendly, benevolent, docile and your perfect servant. Flip it, and it will give you the perfect “yes” or “no” to your question and bring you all the success in life you ever wished for.

Note

I honestly doubt that such an AI exists today or will exist in my lifetime, but this is a thought experiment, so let’s pretend that it could.

There is one catch: I did mention that the coins look identical. You have no access to their decisionmaking internals. You cannot question the coins about their nature. You can flip them as often as you like in response to decision problems.

How do you tell the difference between the two coins?

Info

This thought experiment is closely related to the Chinese room thought experiment proposed by Searle (1980). I believe a coin is closer to the lived reality of most people, though. See also the famous imitation game proposed by Turing (1950) which has been popularized as the Turing Test.

Option 1: Evalute the Coins Yourself Link to heading

You could try to evaluate the coins yourself. Test them against your own problems. If the results match your intuition or seem useful, you should be able to rely on the tested coin as a tool. If you become extremely successful, good-looking, wealthy and sought-after it would seem that your coin is the advanced AI coin. If the results just don’t seem to make a difference in your life, you probably are using the traditional coin.

Unfortunately this personal evaluation strategy isn’t feasible for most high-stakes decision problems.

First, there is usually a significant time lag between decision and result. The results from consequential decisions tend to happen months or years down the line. Examples might be starting a war, founding a startup, investments in the stock market, accepting a new job, the adoption of new laws, modification of administrative practice and so on.

Second, with most interesting real world problems the definition of success is problematic. For example, if someone maybe committed a crime and is judged guilty, is this a case of successful deterrence or a failure of due process? Variation: if someone committed a crime — without reasonable doubt — and is given a decades-long prison sentence, is this a success of retributive justice or a failure to reintegrate the offender into society? You might be able to tell success from failure if you spend a lot of time understanding each individual case, weighing the evidence and considering the advantages and disadvantages of each action in light of your internal moral compass and the values of your society.

If you just flip a coin you would never know the difference.

Third, frequency matters. Sometimes the decision problem only occurs once or is very rare (war, marriage, emigration, choice of education, other existential decisions). Sometimes all the relevant decision problems are similar, but subtly unique (educating students, criminal trials). Most often you will face a couple of related high-stakes problems (perhaps less than ten), but there will be many unique sets of high-stakes problems. Too few to tell if the difference between the coins is a) due to accident or b) due to the true mechanisms powering the tool.

In statistical terms, in the case of important decisions problems your sample size is often too small.

Info

Why is it a problem if your sample size is too small? Check out my open-source tutorial on representativeness, samples and populations for some intuition on sample size. It doesn’t deal with causal inference and statistical comparisons between experimental conditions, but it should be helpful to get a feel for sample sizes.

Option 2: Trust the Expert Link to heading

The alternative is to trust an expert who tells you which coin is all-powerful AI coin.

And this is the crux with modern AI tools: if personal evaluation is infeasible you have to take the word of an (alleged) expert that a) their computer program actually does the super-human intellectual work that it claims to do and b) that it will act in your best interests. Meaning the coin doesn’t just give you a random decision at best or a malicious decision at worst.

Salespeople tell you the similar things when they sell miracle cures or promise you a place in heaven for 999.99 USD. And it still works for much the same reason that astrology, horoscopes, tarot cards, fortune cookies and climate denial remain popular. People want to believe they can access magic, even if there is no rational empirical basis for their belief.

At the end of the day, if you delegate an entire decisionmaking chain it all comes down to whether you trust an unknown person to make future choices for you, your family, friends, colleagues and anyone else you know or don’t know. At the lower end of consequence these are personal choices, at the higher end of consequence these are decisions with life-altering impact for millions of people, such as the passage of laws or the writing of judicial decisions by apex courts.

We spent a lot of time developing democracy and the rule of law to create functional trust arrangements between unknown people. I do not think that kind of trust in computer systems is warranted, if it ever was. The speed at which global tech leaders are willing to bend the knee before autocrats at home and abroad shows that any kind of privately held computerized decisionmaking system will be a future liability to democracy and the rule of law.

Some Objections Link to heading

Objection: Restrict AI to Low Value Decisions Link to heading

Why not stick to low-value decisions? If an AI agent is potentially untrustworthy, just use it for less important tasks or decisions where failure costs little, no?

Honestly, that would be great. Stick to the small decisions. Which restaurant to go to. Which movie to watch. Whether you should have hot chocolate or coffee today. Leave it at that and I’ll get out of your hair.

Sadly, I don’t think that people will leave it at that. Soon we’ll have to fix a lot of important problems left behind by AI agents that have been let loose without properly considering the consequences. We’ll have health insurance claims denied, tax fraud investigations launched, social security benefits revoked and all sorts of other harmful actions taken.

All in the name of artificial super-intelligence.

Objection: Make AI Open Source Link to heading

We could remove the power abuse and corruption angle if every use of AI tools in public decisionmaking was open source, with training data, source code and model weights open to public scrutiny, plus development, testing and production systems controlled by public officials. This seems ideal on first glance, but it is not a practical solution.

First, tax-payer funded institutions almost never have the money to pay market rates for the tech talent required to run government-scale operations at a quality comparable to the top players in the private tech world. The legal profession has spent centuries (successfully) developing its own civic spirit to acquire enough top lawyers to run a country, but even then competition with the private sector remains fierce. The tech sector can look back on half a century of stellar open source work, but its civic-minded sub-communities have not yet reached the critical mass to manage an entire State’s worth of technology as civil servants.

Second, delegating decisionmaking to AI agents controlled by experts — civil servants — is a threat to the separation of powers. Technology will almost always be centrally managed by the executive branch. If the legislative and judicial branches of government rely on decisionmaking agents controlled by the executive, the executive controls the other branches of government. We have seen something similar playing out with expert bills drafted and adopted by the executive, which then sail through parliament without much debate because there literally isn’t a single MP left who understands the specialized technical content of the law.

Objection: Add a Human to the Loop Link to heading

The theme of human-in-the-loop has been all the rage since the phenomenon of “AI hallucinations” gained mainstream attention. Fire up your AI, let it do all the hard work and then have a human double-check the results. You get the best of both worlds: AI speed and human quality. Why not have an AI decisionmaker and let a human double-check the result?

This can’t work. If the human needs to do extensive thinking to come up with the right decision to a problem and the human hasn’t done the required amount of thinking, then the human cannot evaluate the decision with the expected quality.

If the human does all the required thinking to solve the problem, what did you need the AI for in the first place?

Also, humans are lazy. If the computer says no, the human says no. Even if the decision is patently silly. This has been a challenging problem before AI and it will continue to be a problem with AI and after AI.

Objection: Human Agents are Untrustworthy Link to heading

Trust in human agents is also an uncertain thing. Haven’t we all been hoodwinked by a car salesperson, insurance rep, travel agent, lawyer and so on? Since we cannot fully trust humans agents, why is lack of trust in AI agents problematic?

One reason AI agents are fundamentally different from human agents is because control over computer systems is much more centralized than control over humans. The closest analogy to the degree of centralized control that computer systems exhibit are totalitarian States. Popular examples being Nazi Germany, Stalinist Russia and Maoist China. Instead of years of organizing, militancy and purging of enemies, a takeover of an AI agentic system could occur within minutes.

Another fundamental difference is speed. Where a compromised government needs years to make all its civil servants toe the line, a compromised AI system can do so in seconds to minutes. Even a non-compromised but faulty system can make decisions at a rate no corps of human civil servants could match, turning every mistake into a tragedy.

A third difference is that we have developed tools to mitigate the untrustworthiness of humans over the past millennia. These include pro-social evolutionary adjustments, religion, ethics, the rule of law and interpersonal trust policies. With AI agents we are still at the very beginning of these developments. Even if they had similar levels of (un)-trustworthiness as humans, we do not yet have adequate mitigations in place to counter this lack of faith.

Conclusion Link to heading

I began writing my own conclusion to this essay, but Herbert ([1965] 1968, 23) really said everything that needs to be said:

Once men turned their thinking over to machines in the hope that this would set them free. But that only permitted other men with machines to enslave them.

References Link to heading

Herbert, Frank. (1965) 1968. Dune. Hodder; Stoughton.

Reuel, Anka, Amelia Hardy, Chandler Smith, Max Lamparth, Malcolm Hardy, and Mykel J. Kochenderfer. 2024. “BetterBench: Assessing AI Benchmarks, Uncovering Issues, and Establishing Best Practices.” In Advances in Neural Information Processing Systems (NeurIPS). https://betterbench.stanford.edu.

Searle, John R. 1980. “Minds, Brains, and Programs.” Behavioral and Brain Sciences 3 (3): 417–24.

Turing, Alan M. 1950. “Computing Machinery and Intelligence.” Mind 59 (236): 433–60.

For some reason people are quite proud of being told what to do by a random computer program. I find this confusing, to be honest. ↩︎
It’s like people have completely forgotten that internships are supposed to be an educational experience. It has become tragically common see interns only as a disposable source of very cheap labor. ↩︎
And there are enough philosophers that consider free will an illusion and all humans automata by default. ↩︎

A Better Social Media Alternative: Come Join us on Mastodon!

Mon, 10 Feb 2025 00:00:00 +0000

Looking for an alternative to Linkedin, Facebook, Instagram, Twitter/X and all the terrible algorithms that keep hounding you with “AI will eat your job” and “I humbly announce my promotion to master of the world” posts?

You are in luck. Mastodon is a vibrant social media alternative with strong civil society backing that would love to have you.

Want to get started right away? Check out the beginner info over on fedi.tips! And here is my personal Mastodon profile!

I’ve been on Mastodon since 2022 and honestly, it’s been one of the best social media experiences I’ve ever had. You get so much over on Mastodon:

A vibrant, intellectual and avant-garde community
Fantastically interesting posts because people write for people, not to please the algorithm
A modern social media experience with no unnecessary bloat
No advertising
Almost zero AI silliness
Not being blocked by rogue algorithms or insane billionaires
Open source software developed by a non-profit

In terms of user experience Mastodon is closest to the original Twitter. However, you aren’t out of luck of you prefer a different type of social media platform (e.g. Instagram). There are more open alternatives such as Pixelfed (Instagram alternative) or Friendica (Facebook alternative) that can nevertheless interface directly with Mastodon over an open protocol. Collectively they are called the Fediverse (i.e. “federated universe”).

Think of it like cross-posting between Twitter, Linkedin, Facebook and Instagram all the time!

Why not Bluesky? Link to heading

You may wonder: why not Bluesky? Bluesky is a regular venture-backed enterprise. Like Twitter, it will be great for a couple of years, mediocre for a couple more years and then will slowly break down under the pressure of return-on-investment expectations that will destroy everything the platform once stood for.

It’s your choice.

[New Data] Citation Network of the German Federal Court of Finance (2010-2024)

Mon, 27 Jan 2025 00:00:00 +0000

Info

[Open Access] Citation Network of the German Federal Court of Finance as GraphML (2010-2024)
[Open Access] German Federal Court of Finance Corpus (CE-BFH)
[Open Access] Codebook for the CE-BFH Dataset

Overview Link to heading

Following the release of citation networks for the German Federal Constitutional Court (BVerfG) and the German Federal Court of Justice (BGH) I’ve now completed and published the citation network for the German Federal Court of Finance!

The German Federal Court of Finance Corpus (CE-BFH) now includes a specialized variant containing all citations to its own decisions as structured graph data, extracted from the text of those decisions. The variant is still in a beta testing phase.

The scope of the citation network in Version 2025-01-14 is:

ca. 180.000 individual citations
ca. 143.000 edges (citation connections weighted by number of individual citations)
ca. 40.000 nodes (Aktenzeichen and BFHE)

The following types of citations are included:

Citations from Aktenzeichen to Aktenzeichen
Citations from Aktenzeichen to BFHE

Note

“Aktenzeichen” are German docket numbers. “BFHE” are citations to the official collection of decisions.

Aktenzeichen (docket number) citations are less accurate than decision citations (which require a date in addition to the Aktenzeichen for unique identification). That being said, 98,73 % of all Aktenzeichen in the CE-BFH dataset are unique (independent of date), so the Aktenzeichen is a reasonable approximation. I intend to add decision-level citation support in the future.

Warning

Citing decisions can only be those where the full-text is available. This means decisions dated 2010 or later. Cited decisions can be from any year.

Basic Network Statistics Link to heading

Number of Nodes	Number of Edges	Strength (Out)	Mean In-Degree	Max In-Degree	Min In-Degree
39,875.00	143,281.00	178,806.00	3.54	120.00	0.00

Network Diagram for the First, Second and Third Senates Link to heading

These diagram visualize the citation network extracted from the decisions of the First, Second and Third Senate. They represent only a subset of the data. The complete network is probably too large to be visualized in any single diagram.

The visualization algorithm is Sugiyama.

The small white dots are individual docket numbers or BFHE decisions, the connecting lines are citations (any number). Multiple citations between the same nodes are not shown in the diagram, but the weights are available in the network data.

The network is hierarchical, because newer decisions can only cite older decisions, not the other way around. The diagram is read from top to bottom.

Because of the strong connections between certain decision clusters one might call them “lines of jurisprudence”, but the research on this subject is still in its infancy. The shape may simply be an artifact of the force-directed layout.

Info

High-resolution versions of the diagrams can be downloaded here..

Citation Network of the First Senate Link to heading

Citation Network of the Second Senate Link to heading

Citation Network of the Third Senate Link to heading

Technical Note Link to heading

Note

The source code for the dataset and all analysis and quality control results are available open access.

[Essay] The Limits of Reproducibility with Rocker Docker Images for R

Mon, 16 Dec 2024 00:00:00 +0000

Overview Link to heading

Reproducible scholarship is hard, but important for the scientific record and for saving yourself a lot of pain down the line. The same is true for business, because if your software makes you money and it breaks down it stops making you money.

Computational reproducibility requires controlling the code, data and software environment (Buckheit and Donoho 1995; Donoho 2010, 385). Docker is a very helpful but complex solution to reproducibility in R and Rocker Project (Boettiger and Eddelbuettel 2017; Nüst et al. 2020) images make the process easier. However, the r-ver line of images does not version-lock R packages for the latest version of R.

This problem can persist for a long time if one relies too much on the Docker build cache, as I did.

Possible solutions to the Rocker version-lock problem are:

Use an r-ver image with the non-latest R version
Pull images often
Set the CRAN mirror date explicitly in the Dockerfile (my preference)

The failure to ensure reproducibility can lead to catastrophic errors and wild debugging goose chases.

Read on for a thrilling tale of what can go wrong if you don’t read the documentation carefully.

Table of Contents Link to heading

Reproducible Scholarship Link to heading

Those who follow my work know that I spend a lot of time and effort to ensure that my scholarship is reproducible. This quote earned a permanent home in the Code section on my website:

An article about computational results is advertising, not scholarship. The actual scholarship is the full software environment, code and data, that produced the result. [Buckheit and Donoho (1995); Donoho (2010), 385]¹

Reproducibility is not an easy problem and it means different things for different people in different disciplines at different times, but this zinger is a good start.

Most people who teach and advocate for responsibility spend a lot of time getting people to write clear analysis pipelines in computer code and, most importantly, getting scholars to publish that code so others can check if the code really does what it is supposed to do. This is important and we should keep doing this. Especially in Empirical Legal Studies and Legal Data Science this is not yet the default.

Data cleaning and curation is just as important, but it is taught less often and valued less. Everyone likes using “advanced mathematics”, “state-of-the-art statistical techniques” and “groundbreaking artificial intelligence models” for their fancy research, because its just not the same kind of awesome talking about how you wrestled those 20 date formats into nice ISO YYYY-MM-DD and explaining the REGEX that created all those individual variables you fed into your model. We should be taking more about data curation, we should spend more time on it and we should publish all the data we can legally and ethically do, but this is not what this post is about.

This post is about software environments in computational research and how they are important even if everything else is done right. Ensuring a reproducible software environment is partially about principles and supporting the scientific record, but most importantly it is about saving yourself a lot of (avoidable!) trouble down the line.

Note

By the way, computational reproducibility is also a business problem. If you are offering a service or product that earns you money, you want that product/service to keep working and keep making you money instead of randomly breaking down and making your customers look for their pitchforks.

Remember when Crowdstrike crashed half the world in 2024?

The Software Stack in R Projects Link to heading

In modern software there are always dependencies. No one spells out the ones and zeros for the machine by hand, so we write towers of software built on towers of software built on towers of software. And these dependencies change. Upstream authors fix bugs, introduces new features, change the interface, patch security vulnerabilities and so on. This is a good thing, in general.

However, it is a also a problem. If the behavior of dependencies changes, so does the behavior of our own code. Often this is not a problem, but often enough it is.

For data analysis workflows in R these are some of the most important dependency layers:

R packages, e.g. {quanteda} for text analysis
The R language itself
System dependencies of R packages, e.g. Tesseract for {tesseract}
System dependencies of the R language, e.g. the basic linear algebra subsystem
The operating system

Docker Environments and Rocker Images Link to heading

There are many technical solutions to maintaining a reproducible computational environment in R. The core mechanism is to lock software versions of dependencies in place and record any changes. {renv} is a popular option, but it only locks the versions of R packages. {rix} is a new arrival to CRAN that also locks the R version and system dependencies.

My requirements a are a little more challenging so I use Docker to lock the entire software stack from the OS upwards in place. Docker is constructed around a few core concepts:

A container is the actual environment in which the code runs. These should be treated as disposable and re-created daily, hourly or after every run.
An image is the blueprint for creating a container. Creating a container from an image is as simple as reading from the hard drive. Usually takes a few seconds at most.
A dockerfile is the blueprint for creating an image from layered instructions. Building an image usually involves downloading a base image with a particular OS, installing and compiling software and adding config options. This can take minutes or hours.

Obviously, creating a reproducible R software stack from scratch is a huge amount of work, so I rely on the Rocker Project’s Docker images for R (Boettiger and Eddelbuettel 2017; Nüst et al. 2020) as a baseline. The r-ver line of images freezes the OS and R versions, all system libraries at a certain date 90 days past the Ubuntu version release and the R packages at a certain CRAN date linked to the R version.

With this foundation I specify a Dockerfile that installs TeX packages, system dependencies for R, compile Tesseract from source and install R packages. See this Dockerfile for an example. I re-create containers several times a day, but prefer to rebuild the image only if the Dockerfile changes and use the Docker build cache to save time.

The Limits of the Rocker R Package Version Freeze Link to heading

Now, I said that the Rocker images freeze the versions of all R packages at a certain date, as available on CRAN. This is true, but unfortunately not always. The documentation says this:

Non-latest R version images installs all R packages from a fixed snapshot of CRAN mirror at a given date. This setting ensures that the same version of the R package is installed no matter when the installation is performed. (Source)

Note that it says “non-latest R version images”. So with the image containing the latest version of R — even with the reproducible r-ver lines of images — the CRAN mirror will NOT be locked to a fixed date.

Why is this a problem?

When the CRAN mirror is not locked to a date the R package manager will keep installing the latest versions of R packages. See below for how this caused a catastrophic failure in one of my projects.

If the Docker image is built from cache and the base image is not refreshed this behavior can persist for a very long time. This also happened to me.

Solutions to Ensure Date-Lock of R Packages Link to heading

Fortunately there are some easy solutions to the r-ver package version freezing problem. The hardest part is knowing that the problem is even there.

Solution 1: Use Non-Latest R version Link to heading

The easiest and best option is to use an r-ver image with the non-latest version of R.

I recommend this for everyone.

At the time I couldn’t do this because I needed to upgrade to R 4.4.0 quickly because of a major security vulnerability.

Solution 2: Pull Images Often Link to heading

A different option would be to always pull a fresh image (e.g. daily) when running the pipeline. The R version in the project will become “non-latest” after a few months and then fresh images from Rocker will have the package date lock in place. This can be ensured by adding the --pull and --no-cache flags to the docker build command.

My pipelines always attempt to rebuild the Docker image before they run, but I rely on the cache a lot to ensure speedy development. Rebuilding images without before every run would hit me with 15 minute delays all the time.

I don’t recommend this option because re-pulling the base image will waste a lot of time and resources. It will also not version-lock packages in the period where the R version is still the latest.

Solution 3: Set CRAN Mirror Date Explicitly Link to heading

The solution I opted for was to set the CRAN mirror date explicitly in the Dockerfile. The Rocker Project conveniently provides a script for doing so. The following line will do the trick (in this case setting the date at 2024-06-13:

1RUN /rocker_scripts/setup_R.sh "https://packagemanager.posit.co/cran/__linux__/jammy/2024-06-13"

I prefer this option for my personal workflows because:

I might want to adjust the CRAN mirror date without changing the R version
It ensures that the date lock is in place even if I am forced to choose the latest r-ver image

The full Dockerfile, including use of ARG variables for clarity, looks like this:

 1# Build Arguments
 2ARG R_VERSION="4.4.0"
 3ARG R_CRAN_MIRROR="https://packagemanager.posit.co/cran/__linux__/jammy/2024-06-13"
 4
 5# Base Layer
 6FROM rocker/r-ver:${R_VERSION}
 7
 8# LaTeX Layer
 9RUN apt-get update && apt-get install -y \
10    pandoc \
11    pandoc-citeproc \
12    texlive-science \
13    texlive-latex-extra \
14    texlive-lang-german
15
16# System Dependency Layer
17COPY etc/requirements-system.txt /
18RUN apt-get update && apt-get -y install $(cat /requirements-system.txt)
19
20# Tesseract Layer 
21COPY etc/requirements-tesseract.sh /
22RUN sh /requirements-tesseract.sh
23
24# R Layer
25COPY etc/requirements-R.txt /
26RUN /rocker_scripts/setup_R.sh ${R_CRAN_MIRROR} && \
27    Rscript -e 'install.packages(readLines("/requirements-R.txt"))'

Tracing the Error Link to heading

I wrote this essay because this (to me) unexpected behavior of the r-ver Rocker images caused a catastrophic failure in one the new data pipelines I am developing.

The chain of problems that led me there was epic. The exact error isn’t interesting because I wouldn’t claim that this chain of events is particularly common. However, it provides a valuable lesson of how reproducibility failures may cause arcane failures in practice.

So, what happened?

I recently re-built the Docker Image for this new pipeline and was presented with the following error message:

 1The R temporary directory appears to be within a folder mounted as 'noexec'.
 2 Installation of R packages from sources may fail.
 3 See the section **Note** within `?INSTALL` for more details.
 4
 5 tempdir(): /tmp/RtmpL9N978
 6   |............................            |  70% [pipeline-run]               
 7
 8 Error: targets::tar_make() error
 9
10     • tar_errored()
11
12     • tar_meta(fields = any_of("error"), complete_only = TRUE)
13
14     • tar_workspace()
15
16     • tar_workspaces()
17     • Debug: https://books.ropensci.org/targets/debugging.html
18
19     • Help: https://books.ropensci.org/targets/help.html
20
21     _store_ there is no package called ‘qs2’

My chain of thoughts and debugging efforts was something like this:

The first part sounded horrific. Temporary directory mounted as “noexec”? Even though I hadn’t changed anything? This sort of things usually only happens when my hardware is breaking down and that is usually Bad. Capital B.
Keep on reading. No package called “qs2”? This is ok. I manually installed {qs2} in the container and the pipeline re-ran without errors.
But why didn’t the Dockerfile install {qs2} ? In this case {qs2} is a dependency of {targets} (Landau 2021) but it wasn’t installed automatically because it isn’t a formal dependency on CRAN, but an optional dependency for a variant setting that I enabled.
Add {qs2} to the R package list in the Docker config scripts? This is what I did with {qs} and it’s there. But why does the pipeline suddenly require {qs2} ? This can’t be good.
{qs2} became a dependency of {targets} in version 1.9.0, released on CRAN on 2024-11-20. The r-ver image for R 4.4.0 is supposed to be locked to CRAN at 2024-06-13. This is bad.
I checked the R package mirror with options() and it was set to “latest”. Very bad.
Went on a deep-dive to discover why the CRAN mirror in r-ver was set at “latest” and discovered that this was linked to using the latest R version.
But it’s December 2024, and 4.4.0 is NOT the latest R version at this time. So why was my package manager setting still set to “latest”? Turns out I pulled the r-ver image months ago and forgot to re-pull the Docker image when the R version became non-latest.
How to prevent this from happening again? Set explicit version lock in Dockerfile.

Conclusion Link to heading

This problem can persist for a long time if one relies too much on the Docker build cache, as I did.

Possible solutions to the Rocker version-lock problem are:

Use an r-ver image with the non-latest R version
Pull images often
Set the CRAN mirror date explicitly in the Dockerfile (my preference)

The failure to ensure reproducibility can lead to catastrophic errors and wild debugging goose chases.

References Link to heading

Boettiger, Carl, and Dirk Eddelbuettel. 2017. “An Introduction to Rocker: Docker Containers for R.” The R Journal 9 (2): 527–36. https://doi.org/10.32614/RJ-2017-065.

Buckheit, Jonathan B, and David L Donoho. 1995. “Wavelab and Reproducible Research.” In Wavelets and Statistics, 55–81. Springer.

Donoho, David L. 2010. “An Invitation to Reproducible Computational Research.” Biostatistics 11 (3): 385–88. https://doi.org/10.1093/biostatistics/kxq028.

Landau, William Michael. 2021. “The Targets r Package: A Dynamic Make-Like Function-Oriented Pipeline Toolkit for Reproducibility and High-Performance Computing.” Journal of Open Source Software 6 (57): 2959. https://doi.org/10.21105/joss.02959.

Nüst, Daniel, Dirk Eddelbuettel, Dom Bennett, Robrecht Cannoodt, Dav Clark, Gergely Daróczi, Mark Edmondson, et al. 2020. “The Rockerverse: Packages and Applications for Containerisation with R.” The R Journal 12 (1): 437–61. https://doi.org/10.32614/RJ-2020-007.

The formulation is from Donoho (2010), although it is cited as Buckheit and Donoho (1995). ↩︎

[New Dataset] 36,000+ Criminal Law Judgments of the German Federal Court of Justice (1950-1999)

Tue, 26 Nov 2024 00:00:00 +0000

Info

[Open Access] Entscheidungen des Bundesgerichtshofs in Strafsachen aus dem 20. Jahrhundert (BGH-Strafsachen-20Jhd)
[Open Access] Source Code on Zenodo
[Open Access] GitHub Repository

New Dataset available! Link to heading

Tilko Swalve and I are excited to announce the publication of a new dataset!

We collected, cleaned, sorted and published Open Access more than 36,000 criminal law judgments of the German Federal Court of Justice, the Bundesgerichtshof (BGH).

These data are extremely rare. Extensive collections of older judgments issued by German courts are usually only available commercially — if at all. They are especially valuable for the rule of law, because criminal law deeply impacts the rights of citizens. The German Federal Court of Justice has been setting down criminal law guidelines for all of Germany since its creation in 1950.

Unfortunately the German Federal Court of Justice only started publishing its judgments in 2000 and has not done so retroactively for older decisions. Many important precedents and landmark decisions have been denied to the public until now. They were only available to purchasers of expensive commercial subscription services.

We are changing this. Now.

About the Dataset Link to heading

The dataset Entscheidungen des Bundesgerichtshofs in Strafsachen aus dem 20. Jahrhundert (BGH-Strafsachen-20Jhd) is an as-complete-as-possible collection of judgments in criminal matters issued by the German Federal Court of Justice in the period between 1 October 1950 (the founding of the court) and 1 January 2000, the date that the court started publishing decisions online.

We obtained judgments from five of the courts “senates” (panels) for the years 1950 to 1999. A sixth senate existed from 1954 to 1956, but we have no data for this last senate.

We offer the dataset in machine-readable formats TXT and CSV, but also include the original PDF files for traditional legal work.

Note

Please note that the texts in the dataset are only available in German and the dataset accordingly is documented only in German. This is because NLP practitioners working on data in a certain language should be able to speak the language well enough to be able to read the documentation.

Features Link to heading

31 variables
Data model compatible with the Corpus der Entscheidungen des Bundesgerichtshofs (CE-BGH)
Public Domain (CC-Zero 1.0)
Open and platform independent file formats (PDF, TXT, CSV)
Extensive Codebook
Compilation Report explains construction and validation of the data set in detail
Large number of diagrams for all purposes (see the ‘ANALYSIS’ archive)
Diagrams are available as PDF (for printing) and PNG (for web display), tables are available as CSV for easy readability by humans and machines
Secure cryptographic signatures
Publication of full source code (Open Source)

Content of the Dataset Link to heading

By Year Link to heading

Note

We could not extract a plausible date for 2,419 judgments. They are not included in this diagram.

By Senate (Panel) Link to heading

Note

All judgments have panel metadata. This diagram shows the full dataset.

By President Link to heading

Note

“NA” means that no plausible date could be extracted and the judgment therefore could not be mapped to the tenure of a President. This is the case for 2,419 judgments.

Workflow Link to heading

Note

The data pipeline offers the following features:

Clean filenames
Correct rotation, standardize in portrait orientation
Optical character recognition (OCR)
Automated cleaning of OCR errors related to German legal terminology
Extraction of additional variables
Production of ready-to-use ZIP archives
Comprehensive documentation
Automated unit tests and statistical reporting
Cryptographical signatures

Copyright Link to heading

The dataset is released into the public domain under a Creative Commons Zero 1.0 Universal Public Domain Waiver.

[Essay] Git Credential Management

Sat, 23 Nov 2024 00:00:00 +0000

Overview Link to heading

How to access remote Git repositories securely via command line? And where to store the secrets?

These questions led me down a few unexpected rabbit holes and I discovered many things about Git authentication and many reasons for and against each. I was bound to forget all of this if I didn’t write it down. So this is as much a note to myself as it may be useful to others. I am grateful to the many bloggers that have written posts on Git authentication, particularly this post by Danielle Navarro.

To make it short, username/password authentication cannot be used for Git API access on GitHub since 2021 and the major alternatives are personal access tokens (PAT), secure shell (SSH) and web authentication (OAuth). I settled on PATs because they can be given an expiration date, offer fine-grained access control and reflect security-by-isolation principles that I find persuasive.

%%{init: {'theme':'dark'}}%% graph LR l[Local Machine] -->|PAT via Keyring| g[Remote Git Repository] g -->|Read-only| r[Deployment Server] l -->|Direct SSH| r -->|PAT via Cache| g

PATs need to be stored securely. Among the multiverse of Git credential helpers I prefer using the system keyring on my local machine. On Debian Linux this requires some non-obvious setup of libsecret and compilation of the associated credential helper. On Windows and MacOS it is supposed to work right out of the box, but I haven’t worked on Windows for over a decade and have never had a Mac, so I wouldn’t know. The Linux system keyring works well in practice and integrates nicely with my Emacs workflow.

The Git cache is a good alternative for remote deployment servers. The timeout can be customized, something like 30 to 60 minutes seems reasonable. It is important to cache only tightly scoped PATs on remote machines. If the scope is set to single repository and read-only, the damage of token loss should not exceed that of machine compromise.

Above all else, remember AviD’s Rule of Usability:

Security at the expense of usability comes at the expense of security.

In other words: if security is too much of a pain, people will prefer insecurity (including myself).

Table of Contents Link to heading

The Problem Link to heading

Accessing remote Git repositories hosted on services like GitHub to pull, push or do other things programmatically via API requires authentication. Git-the-software (not GitHub-the-service) originally envisioned username/password authentication as the default, but with the advent of more stringent security practices username/password authentication for API access fell into disrepute and was disallowed on GitHub in 2021. Additionally, two-factor authentication has become the norm (and is enforced for open source contributors such as myself) so the old Git authentication flow which can’t handle a second factor wouldn’t work anyway.

Currently there are three dominant authentication options for Git repository access via command line:

Personal access tokens (PAT)
Secure shell (SSH)
Open Authorization (OAuth)

Personal access tokens (PAT) are very long strings of characters that act as a combined username/password replacement, are only used for API access, can be given an expiry date and support very fine-grained access control. PATs are supposed to be transient and replaced often, so that the loss of a token does not compromise the integrity of the entire account. They could be described as temporary limited passwords, albeit with similar secure storage requirements. PATs are an example of the security-by-isolation principle.

Secure Shell (SSH) authentication is a popular alternative to PAT-based authentication. Instead of using a symmetric transient password-like token, SSH authentication is based on asymmetric public/private key pairs. The private key always remains on the machine of the user, the public key is uploaded to the server. The private key proves the identity of the user, the public key can only be used to verify it, nothing else. SSH is primarily a means of controlling a remote server via command line, but it is possible to use it to manage a remote Git repository instead.

Open Authorization (OAuth) is the third major access route to GitHub repositories. At the beginning of a standard OAuth flow a browser window is opened and the user must login to GitHub with their username/password and second factor. Following this a time-limited OAuth token is created and used for subsequent non-interactive authentication.

PATs are very long, are supposed to change regularly and are impractical to memorize, so they must be stored securely — somewhere. The same is true for SSH keys, the private key must also be kept safe. Even with OAuth the OAuth token must be stored safely during its validity period.

This is the crux of the problem: where to store the secrets?

The Multiverse of Git Credential Helpers Link to heading

Over the years the Git ecosystem has added a bewildering array of credential helpers to make it easier to supply Git with the appropriate authentication credentials. Credential helpers don’t store the secrets themselves, but are interfaces to storage options that do.

Below is a list of almost all of the credentials helpers mentioned in the Git documentation, minus the Azure/Netlify specific ones, which are irrelevant to my use case:¹

Native to Git
- Git Credential Store
- Git Cache
Platform Specific Keyrings
- Linux GNOME Keyring or KDE Wallet
- MacOS Keychain
- Windows Credential Manager
OAuth
- Git Credential Manager (GCM)
- Git Credential OAuth
Password Managers
- Gopass
- Lastpass
- 1Password
- KeepassXC

I’ve decided to use Git Cache and the Linux keyring in my personal workflow. This essay includes setup instructions for cache and keyring. My daily workflow looks something like this:

%%{init: {'theme':'dark'}}%% graph LR l[Local Machine] -->|PAT via Keyring| g[Remote Git Repository] g -->|Read-only| r[Deployment Server] l -->|Direct SSH| r -->|PAT via Cache| g

I use the KeepassXC password manager to store all my PATs, so adding a KeepassXC credential helper interface might be useful. However, my specific system setup makes this impractical. I do mention it because others might find this option useful.

Not sure where to fit in the Git Credential Manager (GCM) here. The git-credential-oauth software seems a much better and focused option (400 lines of code vs 40,000 for GCM) for OAuth and GCM secrets storage for PATs appears to be handled by the system keyrings anyway, so might as well use those and save on bloat. I pretty much ignored GCM, although others seem to like it.

I decided against using the native Git Credential Store (cleartext storage! on the disk!) or SSH (many reasons, complicated). Your use case and security considerations may differ, but I’ve written down my reasons below. I’m interested in OAuth, but haven’t gotten around to trying it.

Realistic Option 1: Git Cache Link to heading

The native Git Cache is one of my preferred credential helpers. It can be set up with just a single command:

1$ git config --global credential.helper cache

When an action requires authentication and the PAT is entered, Git stores the PAT in memory, never on disk. By default the timeout is 15 minutes.

The timeout can be configured separately, by adding the number of seconds until timeout in the same command. The following sets a 1-hour timeout:

1$ git config --global credential.helper 'cache --timeout=3600'

Relying on the Git Cache is far more secure than storing the PAT in cleartext on disk, but one should assume that keyloggers would still be able to capture the PAT, maybe even extract it from memory, if the machine is compromised.

This is why it remains important to set an appropriate cache timeout, set the PAT scope narrowly and include a reasonable PAT expiry date to limit damage. Temporal and thematic limitations of the PAT are more important than the cache timeout.

Personally I use the Git Cache for pulling code to remote machines if the project requires more resources than I have available locally. I set the Git Cache timeout to between 30 to 60 minutes.

Nevertheless, the most important security measure is to set tight access controls on the PAT. If the PAT is used on a remote server, I set it to single repository scope and read-only permissions. For the code in the repository to be executed it must be located on the remote machine in cleartext anyway, so losing the PAT should not cause more damage than the machine compromise itself.

Realistic Option 2: System Keyring Link to heading

The keyring is a popular system feature on many platforms that stores usernames and passwords in secure encrypted storage and can only be accessed with the proper credentials, often a primary password, but more recently also biometric features or hardware tokens.

The advantage of the system keyring is much greater convenience compared to the Git Cache in daily usage, since it stores a PAT permanently in encrypted storage. Once the keyring is unlocked, the PAT stays in memory until logout (on Linux).²

The system keyring is also more pleasant to use with Emacs, my preferred text editor. Pushing from Emacs with the git-credential-libsecret helper will open a system prompt to log-in with the keyring, whereas pushing with Git Cache simply fails and forces me to break workflow and enter the PAT on the command line at least once.

The disadvantage of the system keyring on Linux is that once unlocked it stays unlocked until the session is ended. It appears possible to set the keyring timeout somewhere, but so far I haven’t found managed to figure out how. In the meantime this useful command restarts the keyring and flushes the cache:

1$ gnome-keyring-daemon -rd

Configuring the System Keyring Link to heading

Configuring Git to use the system keyring is supposed to be very easy on Windows and Mac. I think it even works out of the box, since credential helpers for the Windows Credential Manager and MacOS keychain are included with Git. I haven’t worked on anything but Linux for a long time, so I wouldn’t know. I do hope it works that way.

Libsecret is an interface between the Linux keyring (the storage for secrets) and other applications that want to access the keyring. There are three steps to installing and configuring Libsecret as the Git credential manager on Linux. It turns out that it isn’t actually difficult, just very much non-obvious.

The three steps are:

Install packages
Compile credential manager
Configure git to use libsecret

Step 1: Install dependencies Link to heading

These are the packages required to make it work, including make and gcc to build from source, git in case it’s not already installed and libsecret.

1$ sudo apt install make gcc git libsecret-1-0 libsecret-1-dev libglib2.0-dev

Step 2: Compile Git-Libsecret-Connector Link to heading

This is the non-obvious part. Git technically already comes with a credential helper that connects to libsecret, but on Debian and Ubuntu it is shipped as pure source code, not as a ready-to-use binary.

It therefore needs to be compiled first. Fortunately, this is very easy if you know the magic words.³

1$ sudo make --directory=/usr/share/doc/git/contrib/credential/libsecret

This compiles the C source code into a usable binary.

Note

A binary for git-credential-libsecret is available for RPM-based distributions.

Step 3: Configure git to use Libsecret Link to heading

The following line sets the Git configuration file to use libsecret to access the system keyring. It works just like the Cache configuration line above.

1$ git config --global credential.helper /usr/share/doc/git/contrib/credential/libsecret/git-credential-libsecret

Now it should automatically trigger the keyring whenever you push, pull or do something that requires authentication!

Why not Git Credential Store? Link to heading

The integrated Git Credential Store is the easiest credential helper to set up and the most convenient to use, but also the most insecure. This is because it saves the PAT in cleartext on disk. Yes, in cleartext with no security whatsoever. I mention the command only because it is the one I choose to avoid:

1$ git config credential.helper store

Anyone who can access the disk can access the stored PAT. The most likely breach scenario would be some kind of drive-by infection with a trojan distributed by an ad-network (ads are everywhere on the internet, after all) that searches the disk for plain-text credentials and steals them.

So, this is a hard no.

Why not Secure Shell (SSH)? Link to heading

SSH is pretty neat, but it does have a a number of problems that decided me against relying on SSH for managing Git repositories: 1) the private key is stored on disk, 2) the key does not expire, 3) the key access controls cannot be scoped (deployment keys are a way around this), 4) SSH forwarding is insecure and 5) the SSH port 22 is often blocked on large managed networks, for example university networks.

Problem 1: SSH key stored on disk Link to heading

In the classic Git Credential Store example the SSH private key is an ordinary file stored on disk. This means that pretty much all the security considerations that apply to cleartext passwords also apply to SSH keys. If the disk is compromised (e.g. drive-by trojan from an infected ad-network) then all SSH keys stored on disk will be collected as fast as any cleartext passwords.

Password for the SSH private key? While it is possible to set a password for the SSH key, the additional protection is limited. It is quite likely that on a compromised machine all command line input, including the password to the SSH key when entered, would be collected and exfiltrated as well.

Storing the SSH private key in the system keyring? Of course it is possible to store the SSH key in the system keyring, same as the PAT. This, however, requires setting up the system keyring and negates any convenience benefits of accessing remote repositories with the simple SSH setup. If the system keyring is in use one might as well go with a PAT.

Problem 2: SSH key does not expire Link to heading

SSH keys do not expire. I don’t think it is possible to set an expiry date on a local SSH keypair and I have found nothing to the contrary so far.

It is possible to set an expiry date for an uploaded public key, if the server supports this. However, GitHub does not support expiry dates for uploaded SSH public keys.

If the key is compromised, this means the key will be compromised for a very, very long time. Which can be very, very bad. The recently uncovered XZ Utils Backdoor was planted by a covert operative who embedded himself in an Open Source project for several years. It’s not unthinkable that a compromised SSH key could be used for mischief many years down the line.

A special problem arises with SSH deploy keys, which are not linked to particular users and do not expire when the user is removed from an organization.

Problem 3: Cannot scope SSH permissions Link to heading

Regular SSH access to Github does not permit fine-grained access control. The SSH connection to Github is allowed to do everything that can be done via API, which can be somewhere between bad and catastrophic if the key is stolen.

Deploy keys are a partial solution to the scoping problem, but they can only be created with single-repository scope. This can be a problem for some use cases, but in my own workflow it would be a workable alternative. That being said, I think many developers use unrestricted SSH keys in their regular workflows.

PATs on the other hand always permit scoping of permissions and can be tailored to specific use cases. It is possible to create a more encompassing PAT for a local secure machine and to create additional limited PATs, for example with read-only access to single repositories for use on a deployment server.

Problem 4: SSH Agent Forwarding is insecure Link to heading

Normally a the private key needs to be present on the machine accessing the remote Git repository. Storing an unrestricted private SSH key on the disk of a local machine is a medium risk, on remote machines it is an unacceptable risk. Deployment keys with single repository scope are an alternative, but may be impractical during development.

SSH Agent Forwarding solves this issue by forwarding the functionality of the SSH agent, but not the private key itself. The SSH Agent Forwarding workflow looks something like this:

%%{init: {'theme':'dark'}}%% graph LR l[Local Machine] -->|Git Push| g[Remote Git Repository] g -->|Git Pull| r[Deployment Server] l -->|Direct SSH| r -->|Forwarded SSH| g

The deployment server is the place where the code stored in the repository is executed, usually some kind of VM in the cloud. If the deployment server is compromised, then the access rights of the SSH agent can be misused for the duration of the SSH connection. This is much better than losing the SSH key permanently, but it is still quite bad because of the universal access rights of regular SSH keys on GitHub.

In terms of security tradeoffs, compromised SSH Agent Forwarding should have a smaller temporal breach scope than losing a PAT. An SSH connection should not be active for more than a few hours at a time, but it might be compromised on more than one occasion if the breach isn’t noticed. A compromised PAT will be valid until its expiry, usually a maximum of 30 days.

On the other hand, a breached PAT with narrow scope (e.g. only the code that is being deployed on the remote server) would cause much smaller losses than the near-full-account access of SSH connections with an unrestricted SSH key. Repository-linked deployment keys are an alternative.

Problem 5: SSH port 22 often blocked on managed networks Link to heading

Accessing remote Git repositories via API is usually done over two protocols: HTTPS or SSH. HTTPS is the regular web protocol and runs on port 443. SSH is normally used for remotely administering servers and runs on port 22.

The main difference: HTTPS is used by regular people, SSH by power users.

Port 443 needs to be open to access any kind of normal webpage, a ton of REST APIs that run over HTTPS and for many other reasons. Blocking it is like blocking the internet, so this is simply not done on large managed networks (e.g. universities, corporations, etc.).

Port 22 on the other hand is used by very few people, usually those with advanced computer skills, usually those who can wreck things on the network. So it is often blocked for security reasons.

Now, SSH can be tunnelled over the HTTPS port, but this requires extra setup and firewall rules or proxy servers may still interfere. Since I spend a good amount of time connected to university networks, PATs over HTTPS are easier.

Why not OAuth? Link to heading

I still need to try the OAuth credential helper. It seems to be an interesting combination of two-factor authentication combined with the Git Cache.

I do wonder if it is more convenient than storing a PAT in the system keyring. I also wonder if it is more secure than a tightly scoped PAT (I didn’t see any scoping options while reading up on it, but may have missed them).

Something to try in the future!

Conclusion Link to heading

This concludes my reflections on a problem I had some time ago: how to access remote Git repositories securely via command line and where to store the secrets.

I use Git Cache and the Linux keyring in my personal workflow. I’m interested in OAuth, but haven’t gotten around to trying it. The essay contains setup instructions for cache and keyring.

I use the KeepassXC password manager to store all my PATs, so adding a credential helper interface might be useful. Unfortunately my specific system setup makes this impractical. It does seem to be a good option, though.

I decided against using the native Git Credential Store because it stores PATs in cleartext on the disk. I decided against SSH because the private key is stored on disk, the key does not expire, the key cannot be scoped as well as a PAT, SSH forwarding is insecure and SSH port 22 is often blocked on large managed networks.

Hopefully this was useful to someone, if only to my future self wondering why I spent all that time thinking about Git authentication.

List compiled from the linked webpage on 22 November 2024. ↩︎
I believe this is true for both GNOME Keyring and KDE Wallet. ↩︎
Don’t ask how many hours of web searching it took me to figure this out. This is the secret of all textbooks and tutorials: you just see the gain, never the pain. ↩︎

[New Tutorial] Representativeness, Samples and Populations

Mon, 04 Nov 2024 00:00:00 +0000

Info

[New Tutorial] Representativeness, Samples and Populations

Overview Link to heading

The representativeness of scientific research and public opinion polls is one of the defining methodological problems of our time. Whether a single study or a series of studies provide us with general information about the world greatly depends on the quantity and quality of the data collected.

If the methods and data of a study are rigorous and convincing in its own context we call it internally valid. If the methods and data allow us to generalize these results to a wider population we call it externally valid.

Key components of external validity are samples, populations and representativeness. Properly drawn random samples with adequate sample size allow inferences from a thousand persons to an entire country. By contrast, badly drawn samples are often biased in unknown and irreparable ways. The quality and quantity of the sampling process are key to the power of modern opinion polls, which are able to predict the outcome of democratic elections within a margin of just a few percentage points.

However, these results are only helpful when the methods are rigorous. The reliance on flawed data in decisionmaking can cause significant and widespread damage. Nowhere is this more acute than in public policy and the application of the law. It is critical for lawyers to understand when results can be generalized and when they cannot.

This tutorial examines the following themes:

The difference between descriptive and inferential statistics
Samples, populations and representativeness as key concepts
The reference class problem as a PR catastrophe for the Munich Security Conference
Fixing the reference class problem with random sampling or post-stratification
Biased opinion polling and the downfall of the Literary Digest
Visual intuition for random samples
Numerical intuition for random samples
The bootstrap procedure for measuring uncertainty at different sample sizes

This tutorial is rather heavy on theory in the first half, but the second half includes a lot of interesting R code you can run yourself. You’ll learn to understand the process, verify the results and make some cool diagrams. The visuals in particular are far more exciting when you create them yourself!

Info

[New Tutorial] Representativeness, Samples and Populations

Recording of the UN Security Council Corpus Launch Event

Wed, 23 Oct 2024 00:00:00 +0000

Info

[Open Access] Recording of the ‘Corpus of Resolutions: UN Security Council’ Launch Event
[Open Access] Slides and Materials for the ‘Corpus of Resolutions: UN Security Council’ Launch Event
[Open Access] Corpus of Resolution: UN Security Council (CR-UNSC)

Recording of Launch Event Presentation Online Link to heading

Did you miss the launch event for the Corpus of Resolution: UN Security Council (CR-UNSC) in May? We’ve got you covered.

The ESIL Interest Group on International Organizations kindly recorded the talks given by myself, Lorenzo Gasbarri (Scuola Superiore Sant’Anna) and Niccolò Ridi (King’s College London) and made them available on YouTube.

The event was chaired by Christiane Ahlborn (Trinity College Dublin).

The slides and materials for my presentation are also available online.

Open Digital Resources for the 2025 Jessup International Law Moot Court

Tue, 08 Oct 2024 00:00:00 +0000

Open Digital Resources for International Law Research Link to heading

Competing in the 2025 Jessup International Law Moot Court?

Over the past few years I’ve created a number of open digital resources for international legal research that will make your life easier and may give you an edge in the competition:

Complete collection of International Court of Justice (ICJ) case law and dissenting/separate opinions in English and French (1946–2023)
Complete collection of Permanent Court of International Justice (PCIJ) case law and dissenting/separate opinions in English and French (Series A, B and A/B)
Complete collection of UN Security Council (UNSC) resolutions in all six official UN languages (1946–2023)
Useful selection of key treaties and important international legal documents in English

Why bother? Link to heading

No more spending hours downloading individual documents from the ICJ website! You can get all of the case law at once, as a single ZIP archive.

All collections are available as PDF files with helpful file names that will make searching for relevant documents much easier.

The ICJ, PCIJ and UNSC collections also come with TXT files and CSV files to run through your favorite search program and do quick full-text searches across the entire universe of these key UN organs.

Want to go really wild? You can also apply modern Natural Language Processing (NLP) methods to these datasets. I’ve written a tutorial on how to do Keyword-in-Context (KWIC) analyses and create X-Ray Plots with the ICJ dataset.

You might even consider feeding the documents to an AI or whatever, but I have some critical thoughts on that.

And the best part? Link to heading

These digital resources are open access, you aren’t forced to share your personal data with me and you don’t have to attend a mandatory training session with people trying to sell you things. Yeah, this is a thing.

[New Data] Citation Network of the German Federal Court of Justice (2000-2024)

Mon, 07 Oct 2024 00:00:00 +0000

Info

[Open Access] Citation Network of the German Federal Court of Justice (BGH) as GraphML (2000-2024)
[Open Access] German Federal Court of Justice Corpus (CE-BGH)

Overview Link to heading

New citation data available! I’ve updated the Corpus of German Federal Court of Justice decisions and it now includes a specialized variant containing all citations to its own decisions, extracted from the text of those decisions.

The scope of the citation network in Version 2024-09-25 is:

ca. 600,000 individual citations
ca. 440,000 edges (citation connections weighted by number of individual citations)
ca. 100,000 nodes (Aktenzeichen, BGHZ und BGHSt)

Note

Aktenzeichen are German docket numbers. BGHZ and BGHSt are citations to the official collections of decisions in civil (BGH) and criminal (BGHSt) matters.

As of now this variant is in beta testing. It contains the following types of citations:

Citations from Aktenzeichen to Aktenzeichen
Citations from Aktenzeichen to BGHZ
Citations from Aktenzeichen to BGHSt

Aktenzeichen (docket number) citations are less accurate than decision citations (which require a date in addition to the Aktenzeichen for unique identification) However, 91.78% of all Aktenzeichen in the CE-BGH dataset are unique (independent of the date) so this is a reasonable approximation. I intend to add decision-level citation support in the future.

Warning

Citing decisions can only be those where the full-text is available. This means decisions dated 2000 or later. Cited decisions can be from any year.

Some Network Statistics Link to heading

Number of Nodes	Number of Edges	Strength (Out)	Mean Degree	Max Degree	Min Degree
101,474.00	441,884.00	593,154.00	8.71	559.00	0.00

Network Diagram for the 11th Civil Senate (Banking Senate) Link to heading

This diagram visualizes the citation network extracted from the decisions of the 11th Civil Senate, also known as the Banking Senate. It represents only a subset of the data. The complete network is probably too large to be visualized in any single diagram.

The visualization algorithm is Sugiyama.

The small white dots are individual docket numbers or BGHZ decisions, the connecting lines are citations (any number). Multiple citations between the same nodes are not shown in the diagram, but the weights are available in the network data.

The network is hierarchical, because newer decisions can only cite older decisions, not the other way around. The diagram is read from top to bottom.

Info

High-resolution versions of the diagrams can be downloaded here..

[New Tutorial] The Importance of Data Visualization (Datasaurus Edition)

Sun, 06 Oct 2024 00:00:00 +0000

Overview Link to heading

Data visualization is an incredibly important component of a data scientist’s toolbox. Not just for communicating the results of an analysis, but also as a sanity check for detecting obvious problems with the data.

The original Datasaurus was created by Alberto Cairo in 2016 as a humorous demonstration of how a silly image can be presented as a serious dataset. Justin Matejka und George Fitzmaurice later published a technique that could produce “serious” datasets from a wide range of base images, resulting in the Datasaurus Dozen.

This tutorial discusses the Datasaurus with its dozen companion examples and explores how they illustrate the limitations of raw data inspection and pure summary statistics.

Info

[New Tutorial] The Importance of Data Visualization (Datasaurus Edition)

The COVID Case Law of the German Constitutional Court (BVerfG) up to July 2024

Sun, 15 Sep 2024 00:00:00 +0000

Info

[Open Access] COVID Case Law of the German Constitutional Court (BVerfG-Corona)
[Open Access] Source Code to the COVID case law dataset

Overview Link to heading

In January 2021 I started publishing a regular compilation of all COVID decisions of the German Constitutional Court, the Bundesverfassungsgericht (BVerfG). Today I published an updated version up to and including 24 July 2024. Now seems like a good time to write a quantitative review of the constitutional court’s COVID case law.

The dataset Corona-Rechtsprechung des Bundesverfassungsgerichts (BVerfG-Corona) is an automated compilation of all BVerfG decisions associated with the novel Corona virus (SARS-CoV-2).

The compilation is based on the Corpus der Entscheidungen des Bundesverfassungsgerichts (CE-BVerfG) and documents all decisions which contain the character sequences “Corona”, “SARS-CoV” or “COVID” in their full text since 1998.

Version 2024-07-24 of the COVID case law dataset contains 142 decisions. The new version contains 13 decisions more than the previous update in February 2023. The data are published in the following formats:

PDF files for traditional legal research
TXT files for Legal Data Science
CSV file containing all keyword hits plus context before and after (KWIC)
Lexical dispersion plots to provide a quick overview of the case law

Even if you don’t practice Legal Data Science at all, being able to download all COVID decisions as PDF files in a clean ZIP archive is neat, eh?

Note

Please note that while this review is in English, both datasets are published and documented entirely in German. I believe that NLP practitioners should have some proficiency in the language they are analyzing, that is, be able to at least speak enough German to read the documentation.

Time Series: COVID Decisions per Year (2020–2024) Link to heading

Public health measures and restrictions of fundamental rights have been discontinued or scaled back significantly in most countries. The same is true for Germany and this is reflected in the number of COVID-related decisions of the BVerfG, which have notably decreased over time.

For this post I extracted the number of COVID decisions by year from the Compilation Report, created a quick table and visualized the data as a bar plot. The data for 2024 are only partially complete (up to July 2024), but the decreasing trend is readily apparent.

During the pandemic I assumed the BVerfG would keep referring to COVID era norms and events for a long time afterwards, but this does not appear to be the case.

On the one hand this lessening need for constitutional intervention may be due to the lifting of almost all public health restrictions, on the other hand the BVerfG provided many useful constitutional guidelines in its landmark decisions. This may allow the lower courts to do most of the heavy lifting from now on.

Hopefully with more determination than during the pandemic.

 1library(data.table)
 2library(ggplot2)
 3
 4dt <- data.table(Year = 2020:2024,
 5                 Decisions = c(63, 41, 24, 11, 3))
 6
 7ggplot(data = dt, aes(x = Year, y = Decisions))+
 8    geom_bar(stat = "identity", fill = "black")+
 9    geom_text(aes(label = Decisions),
10              vjust = -0.5, color = "black", size = 4)+
11    theme_bw()+
12    labs(title = "COVID Decisions of the German Constitutional Court (BVerfG) up to 24 July 2024",
13         subtitle = "Contain the sequences 'Corona', 'COVID' or 'SARS-CoV' in their full text",
14         caption = "Daten: https://zenodo.org/records/13765529 | Seán Fobbe")

Lexical Dispersion Plot Link to heading

Lexical dispersion plots are one of my favorite types of diagram. They are conceptually easy (keyword search), but nevertheless very useful to quickly scan a large corpus for weakly and strongly relevant texts. Weakly relevant texts are those with few hits, strongly relevant texts those with many hits.

Assuming one can find good and (fairly) unique keywords, the search for relevant texts is child’s play. This is especially useful in the legal domain where highly specific language and unique terms abound.

In this case the Corpus der Entscheidungen des Bundesverfassungsgerichts (CE-BVerfG) (contains all published decisions of the BVerfG from 1998 to July 2024) is scanned for the terms “Corona”, “SARS-CoV” or “COVID”. Any of the terms counts as a hit.

Info

Want to try lexical dispersion plots yourself? It isn’t very difficult and I’ve published an English language open access tutorial that guides you through the whole process with an English language corpus!

Note

You can download a high-resolution version of this diagram in PDF and PNG format.

I find it fascinating how such a simple technique can reliably show the five landmark COVID decisions of the constitutional court. These are (translated):

Federal Emergency Brake I: Curfew and Contact Restrictions (19 November 2021)
Federal Emergency Brake II: School Closures (19 November 2021)
Triage in the case of disabled patients (16 Dezember 2021)
Vaccination Certificate (27 April 2022)
EU COVID lines of credit (6 Dezember 2022)

Summary Link to heading

The COVID case law of the German Constitutional Court, the Bundesverfassungsgericht (BVerfG), is now available up to July 2024 in an easy-to-use digital compilation. The time series shows that the number of COVID decisions has decreased sharply in recent years. It is probable that the BVerfG will declare constitutional closure at some point and the pandemic will move from national and constitutional emergency to simply another episode of legal history.

Let us hope that constitutional law will be more prepared for the next pandemic.

[Slides] Introduction to Legal Data Science

Tue, 03 Sep 2024 00:00:00 +0000

Info

[Open Access] Introduction to Legal Data Science (LLI Student Roundtable)
[Open Access] How to Get Started with Legal Data Science (Tutorial)

Introduction to Legal Data Science Talk Link to heading

The slides for my recent talk “Introduction to Legal Data Science” are now online!

The talk was organized around three questions:

What is Legal Data Science?
Why study Legal Data Science?
Where to get started with Legal Data Science?

This presentation provides an introduction to Legal Data Science and hopefully some answers. Legal Data Science is an interdisciplinary approach to the law that unites legal expertise, programming and mathematics to solve quantitative problems and generalize from real-world data.

In a world increasingly built on technology and data analysis it is important to be familiar with at least the basic techniques of data science to maintain one’s intellectual autonomy and remain competitive in the legal job market. Those who dive deeper into the field will discover many exciting opportunities and new career pathways.

The talk was delivered online on 29 August 2024 to the Student Chapter Roundtable of the Liquid Legal Institute. Thank you for the kind invitation!

Be sure to also check out the tutorial that inspired this talk, linked in the info box above!

New Teaching Section

Wed, 31 Jul 2024 00:00:00 +0000

There’s a new top-level section on my website: Teaching Materials on Legal Data Science

This page collects all the materials I’ve created to help people learn more about Legal Data Science. They include an introduction to Legal Data Science, tutorials, selected essays, ideas for advanced learners and some suggestions for teachers.

Legal Data Science is the application of computational statistical methods to the legal domain. In other words, it combines programming, statistics and domain expertise (Conway 2013).

All the materials I create are published under open licenses and I would be glad to see them used wherever they can help — whether for self-study, in class or anywhere else.

Info

Make sure to begin with the tutorial How to Get Started with Legal Data Science if you are a lawyer and have no prior experience with data science.

As new tutorials, essays and other items are published I will add them to this page, sorted by topic. If you are looking for a chronological list, check out my blog for what is new.

[New Tutorial] How to Get Started with Legal Data Science

Mon, 22 Jul 2024 00:00:00 +0000

People often ask me how to get started with Legal Data Science. So I got to work and wrote up a tutorial!

This tutorial offers a basic introduction to the subject and gives reasons why you should not be satisfied with the traditional focus on literary methods in legal methodology.

Most importantly, it offers a large number of Open Access resources to help you on your way, whether you are looking to become a full-fledged Legal Data Scientist or just want to complement your traditional skill set with some computational approaches to the law.

All publications with a link are Open Access and there’s a lot of them!

Check back every once in a while, as I will add more resources when I come across them.

Info

[New Tutorial] How to Get Started with Legal Data Science (Fobbe 2024)

[Essay] The Breaking of a Social Contract, or Why I am Switching to Copyleft Licensing

Wed, 17 Jul 2024 00:00:00 +0000

Overview Link to heading

Effective immediately, I am switching to copyleft licensing for all future software publications. Copyleft means that redistribution of a work or its derivatives must occur under the same or a similar license. This includes new software, new versions of existing software, libraries, active projects, public development versions on GitHub and anything new or current.

I prefer the GNU General Public License v3 (GPLv3), but will decide on a case-by-case basis if another copyleft license is appropriate. I am considering the GNU Affero General Public License (AGPLv3) for certain projects, but haven’t decided yet.

I am committed to backwards-compatible licensing for all past projects and versions. All past and inactive software publications will remain under the licenses they were originally published with. Usually this is MIT No Attribution (MIT-0).

My academic writing will also move to copyleft licensing, usually Creative Commons Attribution-ShareAlike 4.0 International. This website is already licensed under CC BY-SA 4.0 and this will remain as is.

I will continue to release academic datasets under the least restrictive license possible. For example, if the raw data is in the public domain, the finished dataset will be in the public domain. With official legal and political data there are different principles that are important to me, primarily advancing the rule of law and legal scholarship. This requires the unlimited availability of official legal data.

Continue reading for some deeper reflections on digital infrastructure, free software, the death of corporate open source, the (pill)age of AI, the need to resurrect copyleft licensing and where we go from here as a global community reliant on technology.

Table of Contents Link to heading

Software as Infrastructure Link to heading

Software is much more than a fungible product that can be substituted with an alternative at will. After deployment it tends to become a type of infrastructure, with the surrounding social dynamics adapting to the assumptions, affordances and limitations of software systems. This applies to individual workplaces, but also to entire markets, where user expectations are often defined by the leading product.

Consider the Excel Gene Auto-Formatting Saga. For many years the auto-formatting functionality of Excel caused a large number of data entry errors by reformatting gene names as dates and floating point numbers. For example, “SEPT2” which stands for “Septin 2”, is turned into “2006/09/02” (Ziemann, Eren and El-Osta 2016). The problem was first described by Zeeberg et al (2004), almost 20 years ago. Ziemann, Eren and El-Osta (2016) conducted a “programmatic scan of leading genomics journals” and estimated that “approximately one-fifth of papers with supplementary Excel gene lists contain erroneous gene name conversions”. The problem was so bad that the HUGO Gene Nomenclature Committee (HGNC) decided to rename several genes to avoid future data entry errors (Bruford et al 2020).

Microsoft added an option to disable auto-conversion in 2023 (The Verge 2023), but this is a manual option that needs to be changed by the user.

Note

The European Spreadsheet Risk Interest Group (EUSPRIG) maintains an up-to-date list of spreadsheet horror stories.

The kind of market dominance that can force changes even to scientific nomenclature is anything but rare in the software world. The phenomenon of software-as-infrastructure combined with network effects forces society to bend to successful software products. This generates lucrative business opportunities for actors in control of digital infrastructure.

The standard corporate playbook calls for building a mediocre software product that covers an acceptable number of use cases, attempting to win as many users as fast as possible, locking them into a proprietary system and then extracting maximum profit when the costs of switching are unacceptably high to the individual user/organization. The median product is optimized for generating recurring revenue, not user satisfaction or engineering quality.¹

The Microsoft Windows operating system is perhaps the most famous example of this approach, but it is by no means the only one. Microsoft has captured a number of other software markets with successful business tactics over product quality (word processors, spreadsheets, e-mail). More recently, the success of Facebook, TikTok and every other VC-fueled attempt at building Software-as-a-Service (SaaS) platforms have pursued the same vision of creating profitable monopolies by capturing new product categories as they are created. The plan to “build a platform” has become nearly synonymous with “build a monopoly”.²

The private control of infrastructure accompanied by the destruction of competition through monopolistic practices was a tried and tested means of extracting economic rents (unearned revenue) even before the digital age. The United States boasts a colorful history of “captains of industry” or “robber barons” (your choice) making fantastic fortunes from establishing exclusive or quasi-exclusive control over railroads, public transport, communications, real estate, finance and other key social infrastructure. Similar economic patterns have played out all around the world in other times and places.

Several monopolies of the old industrial age were broken up by successful legal challenges brought by the US government in the early 20th century. Unfortunately, the zeal and success rate of anti-monopoly (“anti-trust” in the US) enforcement waned sharply with the rise of the Chicago School in the 1970s. Microsoft in particular managed to avert the US government’s anti-trust challenges to its business practices and was able negotiate a settlement that kept the company intact in the 2000s. This failure of enforcement in the US is significant, because the US became the global hub of software development precisely during the time that competition law was at its weakest. Companies like Microsoft, Google, Facebook (now Meta) and Apple have built such dominance in certain digital markets that they can deny entry to most new competitors and outright buy all the others (e.g. acquisition of Instagram and Whatsapp by Facebook/Meta). Supposedly this benefits consumer welfare, but I am not convinced.

It was during this era of competition law failure, from the 1980s onwards, that civil society took a keen interest in software as infrastructure and developed surprisingly effective means to challenge corporate control of software and secure this new digital infrastructure for society at large. Socially, this effort was led by the Free Software Movement and the Open Source Movement. Legally, the GNU General Public License (GPL) created by Richard Stallman — the “de facto constitution for the Free Software movement” (Tai 2001) — was a milestone that subverted copyright law with the intent to ensure free distribution and re-distribution of software. Technologically, a wide range of high-quality community-supported projects were successful in challenging corporate dominance of the software world, foremost among them Linux. The Free Culture Movement pursued a similar goal for cultural works, developed the Creative Commons licenses and birthed many successful projects, most notably Wikipedia.

Today, many open software and open cultural works are so pervasive that they are fundamentally a part of global infrastructure. For example:

The Linux kernel and its many distributions (e.g. Debian) power almost all servers in the world
Almost all programming languages (e.g. C, C++, Python, R)
The GNU Project and its free software packages (e.g. Emacs, GNOME, GnuPG)
OpenSSL (and forks like LibreSSL) which secure most encrypted internet communication
The Firefox web browser
Wikipedia as the default global encyclopedia³
Specialized blogs that rival professional news outlets in quality and often outperform them in speed (e.g. Krebs on Security)
The open pre-publication of academic research on arXiv

Free Software versus Open Source Link to heading

At this point I should mention the important distinctions between “Free Software”, “Open Source” and “Source Available”, distinctions that are often lost in public discourse. Roughly speaking the key characteristics between the different concepts and their associated social movements are:

Source Available = Public Source Code
Open Source = Public Source Code + Free Reuse/Redistribution
Free Software = Public Source Code + Free Reuse/Redistribution + Copyleft

Open Source is a pragmatic concept and means that the source code for a program is publicly available, the program can be freely distributed and derived works can be freely created and distributed for any purpose. In particular, the Open Source Definition championed by the Open Source Initiative (OSI) forbids discrimination against persons, groups, fields of endeavor, products, other software and particular technologies.

Free Software is a primarily idealistic and philosophical approach that views software as a public good, based on the four software freedoms (Smith 2007). These are:

Freedom to use the software for any purpose
Freedom to change the software to suit your needs
Freedom to share the software with your friends and neighbors
Freedom to share the changes you make

Strictly speaking the definition of Free Software based on these four freedoms does not include copyleft provisions. The main difference between “Free Software” and “Open Source” movements is that the former stresses an idealistic, the latter a pragmatic approach to software distribution. In practice both are often mentioned side-by-side as “Free and Open Source Software” (FOSS) or “Free, Libre and Open Source Software” (FLOSS) to sidestep ideological debates and because in truth the formal definitions are almost identical.

Copyleft means that redistribution of a work or its derivatives must occur under the same or a similar license. The most famous of the copyleft licenses is the GNU General Public License (GPL). For example, the GPL requires that GPL software and its derivatives must be redistributed either under the GPL or AGPL — no exceptions. A similar principle is imbued in Creative Commons licenses with the share-alike quality.

That being said, the practical approach of the Free Software Movement to Free Software is strongly characterized by a copyleft approach to ensure robust enforcement of these freedoms. Copyleft provisions guarantee the software freedoms not just for the direct license recipient, but for any downstream party that might come into contact with the software. Therefore, even if the official definition of Free Software does not include copyleft, I believe it does so in actual practice and should do so in theory as well. When I speak of Free Software I always mentally include the GPL and copyleft.

Source Available is a deceptive marketing tactic that some companies use to promote their product as Open Source, when in fact it complies with neither the Open Source Definition nor the Four Software Freedoms (OSI 2019). Source Available makes no guarantees beyond the public availability of the source code and usually includes some strict obligations that discriminate against persons, groups or technologies, usually to ensure a commercial advantage.

The evolution towards open software infrastructure seems smooth in hindsight, although the process was anything but painless going through it. Starting in the 1970s, the early innocence of the digital age was followed by intense cultural conflict between civil society and commercial interests about who would control the digital infrastructure of this new era, including the terms under which it would be made available to society at large.

The acrimony between civil society and the corporate world over matters like free software, copyright, information sharing and the boundaries of permissible computer use (“hacking”) was intense. Microsoft’s Steve Ballmer famously called Linux “a cancer that attaches itself in an intellectual property sense to everything it touches” (The Register 2001). Before the streaming years, many DVDs came with explicit FBI infomercials threatening criminal prosecution for unauthorized copying. The Computer Fraud and Abuse Act (CFAA) was a cornerstone in handing down heavy prison sentences for any and all vaguely unauthorized uses of computers, including the violation of Terms of Service (TOS), making the violation of a simple contract a felony criminal offense in the United States.

One of the lowest points of the latter part of the era remains the harsh prosecution and subsequent suicide of Aaron Swartz, one of the leaders of the Free Culture Movement and a co-creator of the Creative Commons organization.

Nevertheless, following the tumultuous rise of the World Wide Web it appeared that a social contract had been forged between corporate interests and civil society in the digital space. Individual creators, a wide range of collectives with different levels of organization and a fair number of corporations would openly publish on the internet some of the finest intellectual works the world has ever seen. These were made available for all to read, share and use (within reasonable constraints set by Open Source and Free Culture licenses), in the expectation that many others in the profit and non-profit ecosystems would return this generosity on principle or at least with principled business acumen.

The economically minded might prefer to frame this development in terms of an “information economy”, “sharing economy”, “platform economy” or “creator economy”, but I think that there is more to this system of mutual expectations than can be explained by conceptualizing it as an exchange of goods and services, governed by supply and demand. Many works are shared without the expectation or even viability of economic returns, so a market-based view cannot adequately explain much behavior on the Web. The idea of a social contract is probably closer to the truth.⁴

After a long and bitter struggle it seemed that profit and non-profit interests were finally roughly aligned, with each supporting the other through somewhat different, but ultimately compatible approaches. The Free Software and Free Culture advocates did not vanish, but there was a general feeling, especially among software developers, that times had changed and the earlier acrimony was no longer warranted.

Even Microsoft (!) has changed its tune and put its software muscle to work. It is now 2024 and Windows comes with a special Windows Subsystem for Linux to make using Linux in Windows machines as painless as possible.

Many others believed the same. For example, The Register reported a corporate executive saying:

In an email to The Register, David Habusha, VP of product at WhiteSource, said that the copyleft license was created by the Free Software Foundation in 1985 “to ensure the evil corporations of that time would not be able to use open-source software and then restrict its redistribution.”

But times have changed, he argues. “It is no longer an ‘us’ vs. ’them’ scenario, meaning the open-source community vs. commercial corporations,” he said.

I counted myself among those who believed we had buried the hatchet and as-open-as-possible with permissive licensing was the default way forward in the future. Academics expand the boundary of human knowledge for the sake of humanity, including corporate humanity, right?

The Death of Corporate Open Source Link to heading

People love telling you to enjoy something while it lasts. We may have reached this point with the corporate commitment to open source. Jeff Geerling recently surveyed the terrain and proclaimed that “Corporate Open Source is Dead” (Geerling 2024). I am inclined to believe him.

The list of high-profile casualties in recent years is impressive:

MongoDB invented an entirely new license in 2018, the Server Side Public License, with draconian terms that were unacceptable to almost everyone (MongoDB 2018)
Confluent switched licenses for some components from Apache 2.0 to the Confluent Community License in 2018 (Confluent 2018)
Cockroach Labs switched the license for CockroachDB from Apache 2.0 to the Business Source License (BSL) in 2019 (Cockroach Labs 2019)
HashiCorp switched their default license from Mozilla Public License v2.0 (MPL 2.0) to the Business Source License (BSL/BUSL) in 2021 (HashiCorp 2021)
Elastic switched their default licenses from Apache 2.0 to Elastic License and Server Side Public License (SSPL) in 2021 (Elastic 2021)
Red Hat took over and killed CentOS in 2021 (CentOS 2021), an Open Source project that was competing with Red Hat Enterprise Linux (RHEL), later stopping the publication of RHEL source code and making the inferior “CentOS Stream (…) the sole repository for public RHEL-related source code releases” in 2023 (Red Hat 2023)
Redis switched their default licenses from three-clause BSD to the Redis Source Available License (RSALv2) and Server Side Public License (SSPLv1) in 2024 (Redis 2024)

The incident that hit hardest for me was Red Hat closing down CentOS and scaling back its Open Source commitments, but the last straw was when I learned about Red Hat’s long-running attempts to circumvent the GPL by contractually forcing customers to decide between a) exercising their GPL rights or b) remaining a customer of Red Hat (Software Freedom Conservancy 2023).

I was a long-time user of Fedora and switched to Debian for reasons of stability, but it’s safe to say that I am never going back. There are philosophical reasons, of course, but considering how Red Hat alienated the CentOS community I wonder if even Fedora has a viable long-term future as a community project.

There is nothing inherently wrong with people looking to earn money. There is also nothing inherently wrong with proprietary software, if it is necessary to keep the lights on. In practice it does seem that, according to the White House cyber policy director, companies like Microsoft need to be “dragged kicking and screaming” (The Register 2024) to do the right thing, such as providing basic security tooling.

It is, however, deceptive and unethical to present a project as Open Source when it is not. It is deceptive and unethical to solicit generous contributions from a community with no financial stake in the success of the software and later to revoke the reciprocal arrangement in the hopes of wringing out some extra dollars.

And this brings us to the age of generative AI.

Note

I have serious doubts as to whether “artificial intelligence” (AI) is an honest or even useful term in 2024, but let us call it that for the sake of the people coming from LinkedIn. I briefly considered putting every mention of “AI” in scare quotes, but I was just too damned lazy.

The (Pill)Age of AI Link to heading

It is forbidden to violate copyright; therefore all pirates are punished unless they scrape in large numbers and to the sound of AI trumpets.⁵

It would seem like the corporate world was inspired by Voltaire, but missed the irony.

The Age of AI is the death of corporate open source writ large, magnified to global scale. Fueled by insensible amounts of venture capital since the launch of ChatGPT in November 2022, many for-profit corporations and some non-profit initiatives on the generative AI hype train have been rampaging across the internet and damaging the information ecosystem and public trust in ways that will not be fully understood for years to come. Instead of swindling contributions from a few willing stakeholders, AI companies are pillaging every bit of content they can get their hands on. Technical security measures, copyright laws and data protection regulations be damned.

Mustafa Suleyman, the CEO of Microsoft AI, had this to say in 2024:

I think that with respect to content that’s already on the open web, the social contract of that content since the ‘90s has been that it is fair use. Anyone can copy it, recreate with it, reproduce with it. That has been “freeware,” if you like, that’s been the understanding.

CNBC via The Verge (2024)

Suleyman is not a lawyer and presumably has not watched any DVDs with anti-piracy infomercials in some time, so he can perhaps be forgiven for being ignorant of the finer points of copyright law. Then again, for proper pirates “the Code is more what you’d call ‘guidelines’ than actual rules”.

What bothers me most is his invocation of the social contract. Even a commercial contract implies a reasonably balanced give-and-take. A social contract implies a similarly balanced exchange in society at large. People publish openly on the Web in the expectation that others will return the sentiment and general welfare increases. However, Microsoft and OpenAI only take, they do not give back. The same is true for almost all AI companies.

OpenAI hasn’t published an open model since GPT-2 and is now one of Wall Street’s wolves pretending to be a non-profit sheep. Microsoft remains the archetype of a closed source company. Google is keeping its cards close to its chest. Only Meta has been making waves with its “open” models and offering to provide source code and weights on request. However, this is a deceptive marketing strategy as the Llama license does not comply with the OSI Open Source Definition and whether access requests are honored or denied is opaque. Also, we know what Facebook/Meta is like as a company. This commitment to “open” is likely no more than a maneuver to temporarily undercut the competition. As soon as Meta is ahead it will certainly “do a Red Hat”. We’ve seen this before.

More recently the French company Mistral AI rose to prominence on its claim of being an Open Source company (Mistral AI 2023).Then a massive Microsoft investment in Mistral was disclosed. Conveniently right after the EU AI Act was adopted and the French government had secured concessions on its behalf due to its standing as a good corporate citizen (The Verge 2024). Mistral continues to publish “open weight” models, but has begun excluding commercial use cases from its licenses (e.g. Codestral 22B).

Note

Followers of my work might note that I publish a large number of legal and political data sets, so perhaps this is a case of pot calling kettle black? The difference being that I exclusively republish legal and political data created by public authorities, sourced from official public databases, with clear legal provenance and permission to re-use.

If laws are adopted and judgments are issued in the name of the people, then the people have a right to acquire, analyze, criticize and re-use judgments to become active participants in the rule of law.

While OpenAI and a couple of others had first crack at the Web, it is now open season with thousands of AI companies going at it full force. This wholesale pillage has an ethical, a legal and an economic dimension.

Ethical dimension — Is it acceptable to take a mind-numbing collection of intellectual works, repackage them as AI and give nothing in return? What does this do to the social contract of the Web?
Legal dimension — Is it permissible under copyright and data protection laws to take legally protected intellectual works, compress them into a machine learning model for profit to and give nothing in return?
Economic dimension — What happens to the creator economy if you repackage a large number of intellectual works, do not provide compensation and use the result to compete with the original creators?

Note

This section was inspired by Judith Donath and Bruce Schneier’s essay The Rise of Large-Language-Model Optimization (Donath/Schneier 2024), which I strongly recommend you read. Donath and Schneier have covered most of what I write here with far greater elegance and insight.

I’m not going to spend much time on the ethical and legal dimensions of this problem.

The ethical dimension is a matter of personal integrity and philosophical depth. Both qualities tend to be lacking in the executive suites of companies in the AI space, so I will save myself the effort of building an elaborate argument that will persuade exactly no one. Suffice to say that if you invoke a social contract to take all you can without giving back, you have understood neither the concept of “social”, nor “contract”, but you do know what “privatization” means.

The legal dimension is largely uncharted territory, so we will probably have to wait a few years for results from ongoing litigation and legislative processes to discover whether AI data set preparation, training or model usage count as copyright events and whether companies producing for-profit models can rely on established copyright exemptions such as “fair use”. I’m not a copyright lawyer, so I recommend you look up your favorite pundit for a legal opinion. I do believe this problem will have to be solved with laws, not litigation.

What I do want to focus on is the economics of the problem. The “creator economy” in the modern digital age provides much more freedom to creators than earlier economic eras did, particularly because now there is a much greater selection of viable intermediaries.

As an academic wanting to share your research in the pre-internet era you had a choice between submitting a book or journal article to one of a few publishers, write a technical report in the rare case that an established organization invited you to participate or send honest-to-goodness physical letters to your academic friends.

These days you can post random thoughts to thousands of strangers on social media (e.g. Mastodon, LinkedIn, Facebook,⁶ Reddit, Instagram, TikTok, YouTube), publish very lengthy posts (yes, I know that this essay is too long) on any subject under the sun to a managed blog in minutes, set up your self-compiled blog in however many weeks you need to get Hugo to work, share draft manuscripts instantly on arXiv or Zenodo, record a podcast, dance your research on TikTok (if you are young) or YouTube (if you are old) and do any number of more or less advisable things that will get your work noticed and draw the ire or admiration of your peers.

Myself, I have made good use of Mastodon and LinkedIn and depend greatly on my personal website to share my work. It’s probably fair to say that without social media and blogging I could not have published the research that interested me and gained the audience that I have. My experiences with traditional intermediaries have been mixed, to say it politely.

However, intermediaries remain. They simply have become more colorful, more exciting, more addictive and more algorithmic. This is certainly true of social media, but even academic publishing is becoming more and more like social media every year. Whether you are chasing likes on LinkedIn or citations on Google Scholar, the desire to conform to the expectations of the algorithms is intense. Nevertheless, it is anyone’s guess if today’s algorithms are more opaque than the backroom dealings and ego-driven editorial work of past centuries.

It is important to remember that intermediaries do not just mediate content between a creator and an audience, they also mediate the primary rewards: money and status. Financial rewards can be direct (e.g. subscriptions, shared advertising revenue from platforms) or indirect (e.g. subscriptions, jobs, consulting contracts, paid speaking gigs). Status rewards include short-term recognition (e.g. likes, comments, shares, citations) and long-term recognition (e.g. followers, visibility/mindshare, Board/advisor positions, academic h-index). Status rewards can often be converted into indirect financial rewards with some delay.

A basic model of the creator economy might look something like this:

%%{init: {'theme':'dark'}}%% graph LR Creator -->|Content| Intermediary -->|Content| Audience Audience -->|Money| Intermediary -->|Money| Creator Audience -->|Status| Intermediary -->|Status| Creator

The promise of the old creator economy was that if creators publish content through intermediaries, an audience would reward the intermediary and the intermediary would in turn share the rewards with the original creator. For example, Brian Krebs does great cybercrime journalism, publishes it for free on the Web, an audience looks for current research, Google directs readers to the website and profits from the ads served alongside, while Krebs profits from the additional traffic via ads on his website and by offering paid speaking engagements.

Under the new emerging AI economy this link to the original creator is broken. Instead of directing the user to the original content, AI companies intend to process the entire information request from the audience themselves in order to fully capture the revenue and status streams for themselves. Need current journalism? AI will provide. Need a recipe? AI will provide. Need financial advice? AI will provide. Need a solution to a complicated software problem? AI will provide.

%%{init: {'theme':'dark'}}%% graph LR Creator -->|Content| Intermediary -->|Content| Audience Audience -->|Money| Intermediary Audience -->|Status| Intermediary

There are no language models that can continue to function in the medium and long term without human creators publishing new content for it them repackage. However, in the new AI paradigm the content is identity-laundered and served by the intermediary to compete with the original human creator, starving them of the financial and social rewards they need to continue producing content. Artists are already regretting their generous sharing of images on the Web. Human translation work is dying. Journalists are being laid off by the thousands. More will follow.

Break the link to the creator, break the creator economy. It is economically unsustainable.

However, as Donath and Schneier remind us (Donath/Schneier 2024), there is more at stake here: human connections. Earlier in this essay I wrote that calling the Web an “economy” does it injustice, because there is more at stake than just an exchange of goods and services. Much of what makes the Web so interesting was built with altruistic motives in mind and a simple economic model is insufficient to explain this. The theory of a social contract is much nearer the mark, because the Web is about building a global commons, whether for profit, non-profit or simply as a byproduct of wanting to connect with other human beings.

And breaking the link between humans breaks the entire social contract of the Web.

From Open Source to Free Software Link to heading

It is now clear that the corporate world has broken the social contract of the Web and is doing its best to commodify and re-sell every last part of it. Then again, perhaps this has been happening for much longer and it just took me this long to figure it out.

Note

Have a look at this excellent plea for strong copyleft licensing: On Open Source and the Sustainability of the Commons (Ploum 2024). Ploum argues that this privatization effort has been going on for much longer than just the AI years.

So, where do we go from here? What do we do in the face of AI data pillage and corporate withdrawal from Open Source?

There is, of course, the option to stop sharing any valuable knowledge online. However, I don’t think this is right. The Web has become one of the greatest global commons ever created and this would be like burning down the Library of Alexandria because people have been selling cheap knock-off copies of the books near the entrance.

Technical countermeasures are always an option, but simple ones like robots.txt are routinely ignored by AI scrapers. I can’t say I’m surprised with so much money on the line. Authwalls are not feasible for sites that depend on casual traffic and even big media companies have had trouble generating enough revenue with subscriptions. Some services like Cloudflare offer counter-scraper machine learning-based blocking, but this can have unintended consequences for legitimate traffic. I’ve had to suspend updates to the Corpus of Decisions: International Court of Justice (CD-ICJ) because the Court is now blocking all automated access to its case law.

If history is any guide, this is a battle that has to be fought with legal and political countermeasures. It is also the time to make clear that the tacit social contract has been broken. A number of strategic litigants are already doing admirable work in the courts of law, but this fight needs to be taken into the court of public opinion and the political process as well.

This essay is a small contribution to the ongoing public debate. I think the much greater contribution to the court of public opinion (and a legal countermeasure at the same time) would be the widespread resurrection of viral copyleft licensing.

In the past 15 years or so, copyleft licenses have taken a heavy hit in terms of popularity. The proportion of GPL family licenses used in Open Source projects has dipped significantly compared to permissive licenses (e.g. BSD, MIT and Apache) (Wikipedia 2024). Many have argued that with publicly available software licensing questions had become irrelevant and practicality is now the order of the day. According to Matt Asay, a developer relations manager at MongoDB:⁷

It’s time for the open source Rambos to stop fighting and agree that developers care more about software’s access and ease of use than the purity of its license. (Infoworld 2023)

The opposite is true. In every instance where a company abandoned their OSI-approved Open Source licenses this has led to significant backlash from the relevant community and the forking of the project. With the death of corporate open source and the ravages of AI it seems to me that now is a good time to dust off the old Rambo spirit of Free Software and return to a strong and principled stance on software and content licensing. All the healthiest projects I know (Linux, Debian, Emacs, R, Wikipedia) were built on copyleft licensing, which is a good sign.

I do not mind if people use my published work products in commercial settings (and my chosen licenses explicitly allow this) but a robust reminder is in order that they are intended as part of a global commons to advance the knowledge of humanity and not as exploitable data sludge for greedy executives.

We are still in the opening stages of conflict between creators and AI developers, with claims being staked and the scales of justice being weighted with competing interests. Because of the economics of the problem (and hopefully the ethics, too, but I’m not holding my breath).

I assume that one of two things will happen:

Current copyright laws will be interpreted to cover model preparation, training and deployment
Sui generis rules will be adopted that cover the licensing of intellectual works for model preparation, training and deployment (comparable to the development of database protection rights)

The first outcome is perfectly covered by adopting current-generation copyleft licenses like GPLv3 or CC BY-SA 4.0. So this is the strategy I will adopt for now.

The second outcome will require updating standard licenses to include the to-be-developed rules. I am confident that standard-setting organizations will get behind this. I have much faith that Creative Commons will get this done and much less faith in the Free Software Foundation, but we’ll cross this bridge when we get there.

I have been GPL-curious for a long time, but have held off until I had time to think about the details and long-term implications. Now it is time to make a point. Instead of pragmatic and open distribution there is a need to clarify to the public and to the corporate world that a global commons needs Free Software based on robust and clear expectations that the commons will remain the commons.

Let us return to copyleft licensing to ensure that the social contract of the Web is enforceable.

Bring back the free software and open source Rambos.

Conclusion Link to heading

Effective immediately, I am switching to copyleft licensing for all future software publications. This includes new software, new versions of existing software, libraries, active projects, public development versions on GitHub and anything new or current.

I am committed to backwards-compatible licensing for all past projects. All past and inactive software publications will remain under the licenses they were originally published with. Usually this is MIT No Attribution (MIT-0).

My academic writing will also move to copyleft licensing, usually CC BY-SA 4.0. This website is already licensed under CC BY-SA 4.0 and this will remain as is.

User satisfaction and good engineering do contribute to sales, but they are just two among many factors that decide whether a product is successful in the marketplace. There are outliers in both directions, of course. There exist products with exceptional engineering and user experience offered at a fair price and there are products that are an embarrassment to their developers on top of not meeting any real user needs, despite being sold for hefty sums to top management. ↩︎
Some might argue that most digital markets end up as oligopolies, due to the failure of capturing the complete market. I do wonder if this is not in fact a mischaracterization of the relevant market in many cases. While Facebook and Instagram are technically both social media networks, they serve very different user demographics and in any case belong to the same parent company. ↩︎
Wikipedia has ascended from “don’t trust anything on the internet, anyone can edit Wikipedia!” to “our AI product uses Wikipedia as the primary source of truth and we trust it completely”. ↩︎
You are probably not surprised that a lawyer turns to social contract theory instead of economics, but we’ll get there. ↩︎
This is my take on a famous quote by Voltaire: “It is forbidden to kill; therefore all murderers are punished unless they kill in large numbers and to the sound of trumpets.” ↩︎
Is anyone under 60 still on Facebook? ↩︎
MongoDB specially invented a new software license, the Server Side Public License (SSPL), to secure its commercial interests. The SSPL caused much consternation and shaking of heads in the Open Source community, since using any SSPL components in a service requires publishing the source code for the entire service and all its infrastructure. These terms are so draconian that they are entirely unacceptable to anyone who relies on proprietary components and probably many other users, too. The SSPL was submitted by MongoDB for approval by the Open Source Initiative (OSI), but was withdrawn when it appeared that it would be rejected (OSI 2019). The Board of the OSI later clearly stated that the SSPL could not be considered an Open Source license and called it a “fauxpen source license” (OSI 2019). ↩︎

Slides and Materials for the UN Security Council Corpus Launch Event

Tue, 28 May 2024 00:00:00 +0000

Info

[Open Access] Slides and Materials for the ‘Corpus of Resolutions: UN Security Council’ Launch Event
[Open Access] Corpus of Resolution: UN Security Council (CR-UNSC)

Slides and Materials Online Link to heading

The slides and materials for the UNSC launch event are now available on Zenodo!

The presentation was delivered online on 20 May 2024 during the launch event for the Corpus of Resolution: UN Security Council (CR-UNSC), organized by the Interest Group on International Organizations (IG-IO) of the European Society of International Law (ESIL) and chaired by Professor Christiane Ahlborn (Trinity College Dublin)

Abstract Link to heading

The Corpus of Resolution: UN Security Council (CR-UNSC) collects and presents for the first time in human and machine-readable form all resolutions, drafts, and meeting records of the UN Security Council, including detailed metadata, as published by the UN Digital Library and revised by the authors.

The United Nations Security Council (UNSC) is the most influential of the principal UN organs. Composed of five permanent and ten non-permanent members, its functioning is constrained by the political context in which it operates. During the Cold War, the complex political relationships between the permanent members and their veto powers significantly affected the capacity of the UNSC to address violations of international peace and security, with only 646 resolutions passed from 1946 to 1989. Since the 1990s, the activity of the UN Security Council has increased dramatically and produced 2721 resolutions up to the end of 2023.

In this presentation I provide an overview of the main dataset and its specialized variants, a bibliographic database and the UNSC citation network. I discuss the uses of the UNSC corpus for both traditional and quantitative legal research, including a breakdown of the variable groupings and OCR quality. A special emphasis is placed on the citation data. The slides include many visual representations to provide a quick understanding of the dataset.

Live Demo: Keyword Search Link to heading

The event included a live demo of keyword searches (with context lines) via grep, a well-known command line utility. Grep is included with all Linux systems and can be installed on Windows and Mac. Variants of grep with a graphical user interface (GUI) are available. Free grep GUI tools are AstroGrep and dnGrep.

The Zenodo record includes results from keyword searches suggested by the audience during the launch event. Results are given as a) simple text file with relevant lines extracted and b) HTML files with keywords colored and one context line above/below the relevant line.

Also included is a Bash script to replicate these findings. Instructions:

Download the Bash script from Zenodo
Download the TXT files for Version 2024-05-19
Ensure that the Bash script and a folder containing the TXT files named “CR-UNSC_2024-05-19_EN_TXT_BEST” are in the same location
Open a terminal in this folder and run “bash UNSC_Demo_Keyword-Search_Grep_Script.sh”
All result files should now be in the same folder

Live Demo: Network Analysis Link to heading

Following the keyword search demo I also demonstrated network analysis with Gephi. Gephi is free and open source software with a graphical user interface for interactive network analysis. The GraphML file of the UNSC dataset can easily be imported into Gephi by simply opening it.

The Zenodo record also includes the Gephi project shown during the live demo, with some configuration already set.

For ideas on how to explore the network data I recommend reading through the Gephi manual. Gephi is the easiest way to explore the UNSC citation network data without code.

UN Security Council: Complete Internal Citation Network Published

Tue, 14 May 2024 00:00:00 +0000

Info

[Open Access] UN Security Council Citation Network Data (GraphML Format) (Zenodo 2024)
[Open Access] Corpus of Resolutions: UN Security Council (CR-UNSC) (Zenodo 2024)
[Open Access] Source Code for the ‘Corpus of Resolutions: UN Security Council’ (Zenodo 2024)

UN Security Council Citation Network Data Link to heading

The complete internal citation network of the UN Security Council (UNSC) up to resolution 2722 (1946–2023) with 54 metadata variables is now Open Access and available for download!

It is part of the Corpus of Resolutions: UN Security Council (CR-UNSC)¹

The citation practice of the UN Security Council is special for a number of reasons. First, the text of every UNSC resolution undergoes intense diplomatic negotiations and every word, phrasing and citation is there for a reason. Second, citations to previous resolutions follow a clear, structured and easy-to-extract format. Third, they structure the practice of the UN Security Council into a web of context that is necessary to understand the impact and consequences of individual resolutions.

My colleagues Niccolò Ridi (King’s College London) and Lorenzo Gasbarri (Scuola Superiore Sant’Anna) have published a long and very interesting paper on these questions and the added value that citation analysis can bring to the study of the UNSC:

Ridi, Niccolo, & Gasbarri, Lorenzo. (2023). The role of previous resolutions in the practice of the security council. Columbia Journal of Transnational Law, 61(3), 571-640.

Pre-print is available Open Access via SSRN

How to use the Citation Data Link to heading

Info

[Open Access] UN Security Council Citation Network Data (GraphML Format) (Zenodo 2024)

There are many good network analysis packages for widely used data science programming languages such as R or Python. My favorites for R are {igraph} for analysis and {ggraph} for visualization. You might also like {tidygraph} or {qgraph} .

Igraph is also available for Python.

No Code? No Problem! You can use the citation data without coding skills and with a graphical user interface (GUI) via Gephi. Simply create a new project, open the GraphML file and off you go!²

Visual Representation of the UNSC Citation Network Link to heading

A visual representation of the UN Security Council internal citation network up to resolution 2722. It was created with {ggraph} and a high-resolution version of the image is published in the ‘analysis’ ZIP archive released with the CR-UNSC.

Technical Information on Citation Data Link to heading

The citations were extracted automatically with regular expressions from the English text of the resolutions. Resolutions 1 through 899, 902, 1445 and 1454 are expert-reviewed OCR text and all other resolutions from 900 to 2722 are extracted from born-digital PDF files.

The citation data is published in the widely used GraphML format, distributed within a ZIP archive.

UNSC resolution nodes are associated with as much metadata as makes sense from a network perspective. 54 variables from the main data set are included.

Note that the GraphML file does not contain the full texts of the resolutions to save space.

The main dataset in tabular form is a better choice natural language processing (NLP) tasks. If you wish to combine NLP and citation analysis I suggest you use both in your workflow: the main dataset to select the approriate node set(s), then pass these on to the citation analysis component.

About the Dataset Link to heading

The Corpus of Resolutions: UN Security Council (CR-UNSC) collects and presents for the first time in human and machine-readable form all resolutions, drafts, and meeting records of the UN Security Council, including detailed metadata, as published by the UN Digital Library and revised by ourselves.

The current version collects resolutions 1 (1946) through 2722 (2024) in all six official UN languages (English, French, Spanish, Arabic, Chinese, Russian), with drafts and meeting records in English and a massive number of features.

See this blog post introducing the dataset for an overview of the dataset and its functionality.

Resolution 2722 was adopted in 2024, but I the citation network is complete up to 2023. I mention 2023 here to avoid the impression that the citation network also fully covers 2024. ↩︎
While I do strongly recommend approaching quantitative projects with an open source programming language, release of the source code and reproducible analyses. However, there are a good number of use cases where non-programmers can enhance their own research with network data and the assistance of graphical programs. Following chains of precedent is an important one in legal work. ↩︎

[20 May 2024] Launch Event for the 'Corpus of Resolutions: UN Security Council'

Thu, 09 May 2024 00:00:00 +0000

Info

[Open Access] Slides and Materials for the ‘Corpus of Resolutions: UN Security Council’ Launch Event
[Open Access] Corpus of Resolutions: UN Security Council (CR-UNSC) (Zenodo 2024)
[Open Access] Source Code for the ‘Corpus of Resolutions: UN Security Council’ (Zenodo 2024)

Launch Event Coming Up on 20 May! Link to heading

We will officially launch the Corpus of Resolutions: UN Security Council (CR-UNSC) at an online event on Monday, 20 May 2024, organized by the European Society of International Law (ESIL) Interest Group on International Organisations (IG-IO).

During the event we will give an overview of the dataset, provide background on its construction and offer practical advice on its various uses and limitations, both for traditional and quantitative scholars/practitioners.

You will also have the opportunity to ask the authors any and all questions related to the dataset!

The event will be chaired by Professor Christiane Ahlborn (Trinity College Dublin).

Date: 20 May 2024
Time: 16.30 to 18.00 (CET)
Conference link: https://santannapisa.webex.com/santannapisa/j.php?MTID=mba3098eaebd4ba2e60f4eabc5069db50

Questions about the event? Drop me a message!

Note

The slides and materials are now available on Zenodo.

[New Dataset] The Corpus of Resolutions: UN Security Council

Mon, 06 May 2024 00:00:00 +0000

Info

[Open Access] Corpus of Resolutions: UN Security Council (CR-UNSC) (Zenodo 2024)
[Open Access] Source Code for the ‘Corpus of Resolutions: UN Security Council’ (Zenodo 2024)
[Open Access] Slides and Materials for the ‘Corpus of Resolutions: UN Security Council’ Launch Event

Overview Link to heading

A new dataset is online! And what a dataset it is!

This is joint work with my colleagues Lorenzo Gasbarri (Scuola Superiore Sant’Anna) and Niccolò Ridi (King’s College London) and together we proudly present:

The Corpus of Resolutions: UN Security Council (CR-UNSC)

This is by an order of magnitude the most ambitious and complex legal data science project I’ve ever built and I am very excited for it to finally see the light of day!

We have also included an extensive Codebook that includes a detailed explanation of the dataset and instructions on how to make use of it. Please have a look before using the dataset.

Oh, and do scroll down this post until you see the workflow diagram. It’s worth it, I promise.

Features Link to heading

82 Variables
Resolution texts in all six official UN languages (English, French, Spanish, Arabic, Chinese, Russian)
Draft texts of resolutions in English
Meeting record texts in English
URLs to draft texts in all other languages (French, Spanish, Arabic, Chinese, Russian)
URLs to meeting record texts in all other languages (French, Spanish, Arabic, Chinese, Russian)
Citation data as GraphML (UNSC-to-UNSC resolutions and UNSC-to-UNGA resolutions)
Bibliographic database in BibTeX/OSCOLA format for e.g. Zotero, Endnote and Jabref
Extensive Codebook to explain the uses of the dataset
Compilation Report and Quality Assurance Report explain construction and validation of the data set
Publication quality diagrams for teaching, research and all other purposes (PDF for printing, PNG for web)
Open and platform independent file formats (CSV, PDF, TXT, GraphML)
Software version controlled with Docker
Publication of full data set (Open Data)
Data published under Public Domain waiver (CC Zero 1.0)
Publication of full source code (Open Source)
Free Software published under the GNU General Public License Version 3 (GNU GPL v3)
Secure cryptographic signatures for all files in version of record (SHA2-256 and SHA3-512)

Workflow Diagram Link to heading

About the UN Security Council Link to heading

Under Articles 24 and 25 of the UN Charter, member states have conferred upon the UNSC the “primary responsibility for the maintenance of international peace and security” and have agreed “to accept and carry out” its decisions. The discharge of this function is carried out through the powers bestowed upon it under Chapter VI of the UN Charter, “Pacific Settlement of Disputes”, Chapter VII, “Action with Respect to Threats to the Peace, Breaches of the Peace, and Acts of Aggression”, Chapter VIII, “Regional Arrangements”, and Chapter XII, “International Trusteeship System”.

Under the peace and security mandate, its areas of activity cover disarmament, pacific settlement of disputes, enforcement, and, until 1994, strategic areas in a trusteeship agreement. Its functions also pertain to the correct working of the United Nations, covering issues of membership, the appointment of the Secretary General, the elections of judges of the International Court of Justice (ICJ), the calling of special and emergency sessions of the General Assembly, the amendment of the Charter and of the ICJ Statute.

Updates Link to heading

The CR-UNSC will be updated at least once per year.

In case of serious errors an update will be provided at the earliest opportunity and a highlighted advisory issued on the Zenodo page of the current version. Minor errors will be documented in the GitHub issue tracker and fixed with the next scheduled release.

The CR-UNSC is versioned according to the day of the last run of the data pipeline, in the ISO format YYYY-MM-DD. Its initial release version is 2024-05-03.

Notifications regarding new and updated data sets will be published on my academic website at www.seanfobbe.com or on the Fediverse at @seanfobbe@fediscience.org

Compilation Report and Quality Assurance Report Link to heading

With every compilation of the full data set, an extensive Compilation Report and detailed Quality Assurance Report are created and published in PDF format.

The Compilation Report includes the source code for the pipeline architecture, comments and explanations of design decisions, relevant computational results, exact timestamps and a table of contents with clickable internal hyperlinks to each section.

The Quality Assurance Report contains a count of all hard tests and expectations, additional visualizations and documented test results for all soft tests that require further interpretation

The Compilation Report, Quality Assurance Report and Source Code are published under the following DOI: https://zenodo.org/doi/10.5281/zenodo.7319783

Attribution and Copyright Link to heading

This data is derived from the United Nations Digital Library at https://digitallibrary.un.org. Records were accessed and downloaded on 13 and 26 March 2024, with additional work on revisions and corrections up to and including 3 May 2024.

Pursuant to UN Administrative Instruction ST/AI/189/Add.9/Rev.2 of 17 September 1987 all official records and United Nations Documents (including resolutions, compilations of resolutions, drafts and meeting records) are in the public domain. We wish to honor the letter and spirit of this UN policy. To ensure the widest possible distribution of official UN documents and to promote the international rule of law we waive any copyright that might have accrued by creating the dataset under a Creative Commons CC0 1.0 Universal (CC0 1.0) Public Domain Dedication.

Contact Link to heading

Did you discover any errors? Do you have suggestions on how to improve the data set? You can either post these to the Issue Tracker on GitHub or contact me via https://seanfobbe.com/contact/

[New Tutorial] Distributions and Summary Statistics

Mon, 26 Feb 2024 00:00:00 +0000

Info

[Tutorial] Distributions and Summary Statistics (Fobbe 2024)

RSS Feeds on this Website and Elsewhere

Thu, 15 Feb 2024 00:00:00 +0000

About RSS Feeds Link to heading

Do you read my blog semi-regularly and keep refreshing the article list to check if I published something new? Let me introduce you to RSS feeds:¹

RSS (RDF Site Summary or Really Simple Syndication) is a web feed that allows users and applications to access updates to websites in a standardized, computer-readable format. Subscribing to RSS feeds can allow a user to keep track of many different websites in a single news aggregator, which constantly monitor sites for new content, removing the need for the user to manually check them. News aggregators (or “RSS readers”) can be built into a browser, installed on a desktop computer, or installed on a mobile device.

Source: https://en.wikipedia.org/wiki/RSS

Why should you consider using RSS?

Aggregate as many information sources and as many topics as you like
Get a better cross-section of the news than what strangers’ likes and machine learning can deliver
Don’t lose your feed when a platform dies (Exhibit A: Twitter)²
Avoid the outrage machine of modern social media³
Avoid having your feed gamed by bots
Avoid much of the vapid, self-congratulatory and artificial posts that are popular in machine learning-directed social media feeds

With the current decline of traditional social media platforms, RSS feeds may be experiencing something of a Renaissance. Maybe it’s just me, but people once more seem willing to take the time to curate their own news feeds instead of relying on Twitter et al. to serve them up the news of the day.

It takes some time to curate your own collection of RSS feeds, but it will probably remain relatively stable and useful for several years (excluding a change in your interests and the occasional blogger quitting).

My RSS Feeds Link to heading

If you took a close look at the icons on the front page, you may have noticed that the rightmost icon is the RSS feed icon. It contains a link to the RSS feed for all the posts of this blog.

And not just titles or short descriptions! Unlike many commercial offerings I publish the full text of each post via the RSS feed.

Unfortunately I can’t promise that the occasional diagrams, math and code in my posts will render well in your RSS reader, but neither do they in mine (Newsboat is plain-text only). For those posts there is usually an option to open a regular browser and visit the website.

There are two RSS feeds, one for each language version:

English RSS Feed: https://seanfobbe.com/posts/index.xml
German RSS Feed: https://seanfobbe.com/de/posts/index.xml

You can simply add these to the RSS reader of your choice. For example, the e-mail client Thunderbird supports RSS natively. There are many, many more free RSS readers out there on the web, just spend a few minutes searching for them.

Personally, I use the command line RSS reader Newsboat. I like its no-nonsense approach to news very much, but I’ll freely admit it’s probably not mainstream material.

By the way, this is a zero tracking website. Neither the blog nor the RSS feed track whether or how you access any of the content on this website. The only info I see is aggregate monthly traffic in MB and HTTP hits, which tells me nothing. As it should.

Finding more RSS Feeds to Follow Link to heading

So, you like RSS but aren’t keen on setting up new software just for one blog? Fair enough. There are millions of other feeds on the web. But how do you find them?

News websites (e.g. BBC or The Guardian) almost always offer an RSS feed with their content, usually a headline, short description and link to the main article. Some news outlets also offer full-text RSS feeds for subscribers.
Any website created with WordPress has an RSS feed built-in. You can access it by adding /feed to the end of the URL. For examples and details see the WordPress RSS Feed help section. WordPress is supposed to power around 40% of the web, but regardless of the numbers it’s a lot of websites.
Static website builders like Hugo also create RSS feeds. In this case you have to add /index.xml to the blog URL, for example: https://seanfobbe.com/posts/index.xml
My own blog roll is available via GitHub. It contains a list of URLs, tags and comments that are Newsboat-compatible, but you can also just copy & paste them into your own feed reader.

Not be confused with the Royal Statistical Society, who also go by the acronym ‘RSS’. ↩︎
To be entirely fair, you can also lose your Mastodon account if your instance unexpectedly goes down, but you can prevent this by making regular backups. I doubt that most users are comfortable with doing their backups themselves, but at least the functionality is not terribly difficult to operate once you have located it. ↩︎
Traditional media does have its own outrage mechanics, but you can keep this contained in individual feeds and stack your overall feed with fun science blogs, knitting tips or whatever you are currently into. ↩︎

[New Tutorial] Keyword in Context Analysis (KWIC) and Lexical Dispersion Plots

Wed, 07 Feb 2024 00:00:00 +0000

Info

[New Tutorial] Keyword in Context Analysis (KWIC) and Lexical Dispersion Plots (Fobbe 2024)

[Essay] Visualizing Permutation Matrices as Graphs

Sun, 04 Feb 2024 00:00:00 +0000

Overview Link to heading

Switching the rows of a matrix by multiplying the matrix with a compatible permutation matrix is an elementary operation in linear algebra. This computational essay shows that interpreting a permutation matrix as a transposed graph and plotting it provides a visual understanding of what the permutation matrix will do.

Visualization may aid in understanding the purpose of permutation matrices, show an interesting link between linear algebra and graph theory, as well as provide an introduction to working with matrices in R.

I assume this post will be primarily of interest to beginners in linear algebra or graph theory, but I originally wrote it to explore permutation matrices for myself.

Permutation Matrices Link to heading

One of the elementary operations in linear algebra is the switching of rows in a matrix. This is particularly useful when solving systems of linear equations through Gaussian Elimination based on the classic $ LU $ factorization.¹

Let us assume we wish to solve the following system of linear equations:

$$ \begin{align} 2y + 3z &= 2 \\ 3x + 5y + 6z &= 1\\ 9z &= 3 \\ \end{align} $$

We can then construct a square matrix $ A_1 $ to represent the coefficients of the left-hand side of this system of equations:

$$ A_1 = \begin{bmatrix} 0 & 2 & 3 \\ 3 & 5 & 6 \\ 0 & 0 & 9 \\ \end{bmatrix} $$

In the course of an $ LU $ factorization we need to find an upper-triangular matrix $ U_1 $ with non-zero pivots (the numbers on the diagonal) and all zeros below the diagonal.² Therefore, in an upper-triangular matrix $ U_1 $ the value in the diagonal at position $ (1,1) $ cannot be zero as it is at the moment.

Info

Matrix Index Notation

The notation for the position of a value in a matrix is (row, column). The index $ (2,1) $ means the value in row 2, column 1.

To achieve this we will want to switch the first and second rows to give the first row a non-zero pivot and save us the bother of performing further row operations on the second row. We can accomplish the switching of the first two rows with the following permutation matrix $ P_1 $:

$$ P_1 = \begin{bmatrix} 0 & 1 & 0 \\ 1 & 0 & 0 \\ 0 & 0 & 1 \\ \end{bmatrix} $$

Our permutation matrix $ P_1 $ does the following:

A value $ 1 $ in position $ (2,1) $ when multiplied with another matrix $ A_1 $ moves row 1 of $ A_1 $ to row 2.
Correspondingly, a value $ 1 $ in position $ (1,2) $ when multiplied with another matrix $ A_1 $ moves row 2 of $ A_1 $ to row 1.
The value $ 1 $ in position $ (3,3) $ means that row 3 stays where it is

Info

Permutation Matrices

The values in a permutation matrix are all $ 0 $, except for a single $ 1 $ per row. The placement of the $ 1 $ values determines the type of permutation. Permutation matrices are derived from an identity matrix with the rows switched around, which is also what the permutation matrix does when pre-multiplied with another matrix: switching rows.

In terms of a pre-multiplied permutation matrix the value at a particular index position means (destination, source) of a row operation.

Post-multiplying a matrix with a permutation matrix changes its columns — matrix multiplication is not commutative!

In a nutshell: our matrix $ P_1 $ switches rows 1 and 2 with each other and row 3 remains where it is. When we multiply matrix $ P_1 $ with matrix $ A_1 $ we obtain:

$$ P_1A_1 = \begin{bmatrix} 0 & 1 & 0 \\ 1 & 0 & 0 \\ 0 & 0 & 1 \\ \end{bmatrix} \begin{bmatrix} 0 & 2 & 3 \\ 3 & 5 & 6 \\ 0 & 0 & 9 \\ \end{bmatrix} = \begin{bmatrix} 3 & 5 & 6 \\ 0 & 2 & 3 \\ 0 & 0 & 9 \\ \end{bmatrix} = U_1 $$

Warning

Matrix multiplication is not commutative. $ P_1A_1 $ is not the same as $ A_1P_1 $ !

In terms of the original system of linear equations we would now be solving this form, which presents a more obvious path to a solution:

$$ \begin{align} 3x + 5y + 6z &= 1\\ 2y + 3z &= 2 \\ 9z &= 3 \\ \end{align} $$

With the back substitution algorithm we first solve the last row and obtain $ z = \frac{1}{3} $. We insert $ z $ into the second row and find that $ y = \frac{1}{2} $. Inserting $ y $ and $ z $ into the first row we conclude with $ x = -\frac{7}{6} $.

Note that this is simple example to explain a particular technique and as humans we could have solved the original system of linear equations directly without switching any rows. In cases that include other types of row operations this would not have been possible.

Load Igraph Link to heading

Now to visualize the permutation matrix. For analyzing and visualizing graphs we will rely on {igraph} , one the most comprehensive network analysis packages available in R. While I usually use {ggraph} to visualize graphs, in this simple example {igraph} will do just fine.

1library(igraph)

1## 
2## Attaching package: 'igraph'

1## The following objects are masked from 'package:stats':
2## 
3##     decompose, spectrum

1## The following object is masked from 'package:base':
2## 
3##     union

Input Permutation Matrix $ P_1 $ Link to heading

First we input the permutation matrix into R. By arranging the code a little we can code the matrix almost as we see it.

Nevertheless, it is a good idea to print the stored matrix to make sure the input code is correct. Hard-coding matrices can easily go wrong when columns and rows are switched accidentally. Or when we set nrow or byrow incorrectly.

1P1  <- matrix(c(0, 1, 0, 
2                1, 0, 0, 
3                0, 0, 1),
4              nrow = 3,
5              byrow = TRUE)
6
7print(P1)

1##      [,1] [,2] [,3]
2## [1,]    0    1    0
3## [2,]    1    0    0
4## [3,]    0    0    1

Visualize Small Permutation Matrix $ P_1 $ Link to heading

Now that we have the matrix loaded into R, we create a graph from it. This trick works because a graph can be expressed as a square matrix, so a square matrix can again be interpreted as a graph. The technical term in graph theory is adjacency matrix.

Here we treat the permutation matrix as a directed graph (a ‘digraph’) and transform it first into a graph object, then calculate the transpose, then plot it.³

Note that most of the plotting code below is just to align the optics of the plot with the optics of my website, a simple plot(g1t) call would do just fine.

 1set.seed(999)
 2
 3g1 <- graph_from_adjacency_matrix(P1, mode = "directed")
 4g1t <- t(g1) # Transpose
 5
 6par(bg = "#212121")
 7
 8plot(g1t,
 9     vertex.size = 40,
10     vertex.color = "white")

As we expected, the plot shows that rows 1 and 2 are to be switched, while row 3 is left unchanged.

Note

Graph Transpose

The transpose $ G^T $ of a directed graph $ G $ is just the directed graph with all its edge directions reversed.

Why do we use the transpose $ G^T $ instead of plotting $ G $ directly? This is because igraph adopts the graph theoretic and social science convention of using the row index to describe the source node and the column index for the destination node. Pre-multiplied matrix permutation uses the column index of a permutation matrix as the source row in the target matrix and row index as the destination row in the target matrix. That is why we need to switch the notation with transposition.

In this small example it does not make a difference, since we are just switching two rows with each other. In the more complicated rotating case below it does make a difference.

We could also have transposed the matrix first before plotting it, but this is equivalent to transposing the graph and a graph transpose can be interpreted visually (reversing edges), which is after all what this essay is about.

Large Permutation Matrix $ P_2 $ Link to heading

The previous matrix $ P_1 $ was fairly small ( $ n = 3 $ ), but it is interesting to try the exercise with larger matrices, as the advantage of visualization increases with size. The matrices $ P_2 $ and $ A_2 $ are now of size $ n = 5 $ and look a lot more intimidating:

$$ P_2 = \begin{bmatrix} 0 & 0 & 0 & 1 & 0 \\ 0 & 0 & 1 & 0 & 0 \\ 0 & 1 & 0 & 0 & 0 \\ 0 & 0 & 0 & 0 & 1 \\ 1 & 0 & 0 & 0 & 0 \\ \end{bmatrix} \quad A_2 = \begin{bmatrix} 0 & 0 & 0 & 0 & 25\\ 0 & 0 & 13 & 14 & 15\\ 0 & 7 & 8 & 9 & 10\\ 1 & 2 & 3 & 4 & 5\\ 0 & 0 & 0 & 19 & 20\\ \end{bmatrix} $$

Note

Matrix Size

The concept of a ’large’ matrix is relative. Calling a square matrix of size $ n = 5 $ ’large’ would be silly in the context of Large Language Models (LLM), but is a fair description when performing calculations by hand or hard-coding the matrix by hand, like yours truly had to do when writing this essay. Coincidentally, $ n = 5 $ is also the maximum matrix size I could nicely fit onto a line to show multiplication of the full matrices without sidescrolling.

Input Large Permutation Matrix $ P_2 $ and Matrix $ A_2 $ Link to heading

We input the two matrices into R and print them to verify that they have been coded correctly:

1P2  <- matrix(c(0, 0, 0, 1, 0, 
2                0, 0, 1, 0, 0, 
3                0, 1, 0, 0, 0, 
4                0, 0, 0, 0, 1, 
5                1, 0, 0, 0, 0),
6              nrow = 5,
7              byrow = TRUE)
8
9print(P2)

1##      [,1] [,2] [,3] [,4] [,5]
2## [1,]    0    0    0    1    0
3## [2,]    0    0    1    0    0
4## [3,]    0    1    0    0    0
5## [4,]    0    0    0    0    1
6## [5,]    1    0    0    0    0

1A2 <- matrix(c(0, 0, 0, 0, 25,
2               0, 0, 13, 14, 15,
3               0, 7, 8, 9, 10,
4               1, 2, 3, 4, 5,
5               0, 0, 0, 19, 20),
6             nrow = 5,
7             byrow = TRUE)
8
9print(A2)

1##      [,1] [,2] [,3] [,4] [,5]
2## [1,]    0    0    0    0   25
3## [2,]    0    0   13   14   15
4## [3,]    0    7    8    9   10
5## [4,]    1    2    3    4    5
6## [5,]    0    0    0   19   20

Visualizing the Large Permutation Matrix $ P_2 $ Link to heading

We proceed as we did with the smaller matrix, by creating a graph from an adjacency matrix, then transposing and plotting it. We see that the permutation matrix:

Rotates rows 1, 4 and 5
Switches rows 2 and 3 with each other

1g2 <- graph_from_adjacency_matrix(P2, mode = "directed")
2g2t <- t(g2) # Transpose
3
4par(bg = "#212121")
5
6plot(g2t,
7     vertex.size = 40,
8     vertex.color = "white")

Matrix Multiplication Link to heading

To round off this post, let us multiply the matrices $ P_2 $ and $ A_2 $ to make sure that the permutation matrix does exactly what we expect it to do, namely the following:

$$ P_2A_2 = \begin{bmatrix} 0 & 0 & 0 & 1 & 0 \\ 0 & 0 & 1 & 0 & 0 \\ 0 & 1 & 0 & 0 & 0 \\ 0 & 0 & 0 & 0 & 1 \\ 1 & 0 & 0 & 0 & 0 \\ \end{bmatrix} \begin{bmatrix} 0 & 0 & 0 & 0 & 25\\ 0 & 0 & 13 & 14 & 15\\ 0 & 7 & 8 & 9 & 10\\ 1 & 2 & 3 & 4 & 5\\ 0 & 0 & 0 & 19 & 20\\ \end{bmatrix} = \begin{bmatrix} 1 & 2 & 3 & 4 & 5\\ 0 & 7 & 8 & 9 & 10\\ 0 & 0 & 13 & 14 & 15\\ 0 & 0 & 0 & 19 & 20\\ 0 & 0 & 0 & 0 & 25\\ \end{bmatrix} = U_2 $$

We multiply $ P_2 $ with $ A_2 $ in R to obtain:

1U2 <- P2 %*% A2
2print(U2)

1##      [,1] [,2] [,3] [,4] [,5]
2## [1,]    1    2    3    4    5
3## [2,]    0    7    8    9   10
4## [3,]    0    0   13   14   15
5## [4,]    0    0    0   19   20
6## [5,]    0    0    0    0   25

This result matches what we derived from the visualization of the permutation matrix. It worked!

Conclusion Link to heading

It is possible to transform a permutation matrix into a graph, then transpose and plot it to understand what the permutation will do. This may aid in understanding the purpose of permutation matrices, show an interesting link between linear algebra and graph theory, as well as provide an introduction to working with matrices in R.

Note that this post is intended to provide a better understanding of the concept of permutation matrices. There exist efficient software implementations for Gaussian elimination that can be used out of the box and do not require an understanding of the details of matrix multiplication. ↩︎
Square matrices of this form are called upper triangular. Upper triangular matrices are also always in row echelon form, which is the more generalized description of all matrices that look like (inverted) staircases, not just square ones. ↩︎
An adjacency matrix can also be treated as an undirected graph or include weights. Here we only interpret it as an unweighted directed graph. ↩︎

[Essay] Against the AfD

Tue, 16 Jan 2024 00:00:00 +0000

The AfD Deportation Conference Link to heading

If you’ve been reading the news, you will have heard that Germany is at a boiling point right now, as the German far-right met to plan ‘mass deportations’. The event was located not very far from the original Wannsee Conference, during which the mass deportation of Jews was planned during the Holocaust.

I wrote a long and angry essay about this, arguing for a ban of the extremist AfD party, which was at the center of this conference and is posed to achieve somewhere between 20% and 30% of the vote in some of the German federal states in 2024 and in the federal elections coming up in 2025. We’ve seen this before in German history and it ended badly.

Info

[Essay] Wider die AfD: Eine Streitschrift (Fobbe 2024)

The full essay is available in German only, but I have translated the executive summary for you below.

Against the AfD: Exective Summary Link to heading

We’re taking a break from the regular Data Science programming for a serious national crisis in German democracy. It is the year 2024 and the Federal Republic of Germany must once again confront one of the most awful questions since 1945:

“Pray tell, what are we to do with the Nazis?”

It is widely known that the right-wing and extremist Alternative für Deutschland (AfD)¹ party has been openly promoting inhuman ideas for years, its state-level sections in Thüringen, Sachsen und Sachsen-Anhalt are classified as “proven right-wing extremist” and the leading figure of the party is a Court-approved Nazi and fascist. As with the legend of the frog who does not notice the slowly rising water temperature while being cooked alive, German public discourse has only formed a lukewarm consensus on how to deal with the AfD.²

It is often said that the AfD should be defeated in the marketplace of ideas. But it is now 2024 and who hasn’t yet gotten the memo that Nazis are bad people and the Grundgesetz (Basic Law) is an “Anti-Nazi-Constitution” (Steinbeis)? The actual question is this: is the Nazi level of the AfD significant enough to have transformed them from harmless idiots to dangerous idiots in order for us to activate all political and legal measures of the wehrhafte Demokratie (robust democracy) against them?

The restraint of the political establishment in Germany has so far been based on a reading of the AfD as a party suported by protest voters and establishment skeptics. Dissatisfaction is permitted under the Grundgesetz. Harsh criticism of the government and its policies as well. Thus, under the constitution it is permissible to agitate against European unification and immigration. The AfD was originally founded as an anti-euro party and was able radicalize itself under this cover for a long time. In contrast to the National Democratic Party of Germany (NPD), it was at least partially able to preserve the appearance of a right-wing conservative party, although the ideas and actions of its followers suggest much worse.

This indecisive wait-and-see attitude of public discourse was shattered by new research published by the journalistic network Correctiv: “Secret Plan Against Germany”. The teaser reads:

High-level AfD politicians, neo-Nazis and rich executives met in a hotel near Potsdam in November. They planned nothing less than the expulsion of millions of people from Germany.

We have reached the boiling point. There is no longer just evidence of the extremism of individual state-level units and officials. There are credible signs that the federal AfD and powerful supporters are preparing measures that are reminiscent of the darkest Nazi era. The meeting was held in a location hardly eight kilometers from the Wannsee Conference of 20 January 1942, during which the deportation of European Jews was planned.

It is time to take action against the AfD and activate the robust democracy provisions of the constitution against the party. This includes a party ban (Art. 21 (2) GG), a prohibition on financing the AfD (Art. 21 (3) GG) and the loss of political rights (including the ability to be elected and serve in public office) for key AfD leaders (Art. 18 GG, § 39 BVerfGG). A party ban will not fail for the same reasons as the proceedings against the NPD because it is probable that no government spies are in AfD leadership positions and the AfD poses a significant threat to democracy in truth.

Legal measures must be accompanied by intense efforts of democracy defenders in civil society, politics and business because otherwise the establishment parties will not find the courage to petition the Constitutional Court and go through with proceedings against the AfD.

Public trust in democracy is upheld by the great majority of democratic voters, not by AfD followers alone. Constitutional proceedings to ban the AfD will not weaken trust in democracy, but strengthen it.

“Alternative für Deutschland” translates to “Alternative for Germany”. These days it probably means an alternative constitution based on racial hatred. ↩︎
The story of the boiling frog is an urban legend. Based on current scientific research even frogs will jump out of the water as soon as the temperature becomes uncomfortable. ↩︎

Tags, Categories and Language Versions on this Website

Fri, 05 Jan 2024 00:00:00 +0000

Overview Link to heading

As this website has grown, it has become more important to find efficient ways to organize and manage the increasing number of blog posts and individual pages. Both for the sake of readers and for myself when managing the site.

Hugo, the website builder engine underlying {blogdown} and therefore this website, provides two types of built-in taxonomies to help manage this problem: tags and categories.

While I’ve added tags to posts from the very beginning, I recently started using categories and decided on how each of these should be used. This post collects and explains some of the details for interested readers, but also serves as a reminder to myself in case I forget what I had in mind.

Multilingual Mode Link to heading

Beyond tags and categories, this website is actually two websites in one: an English and a German version. Hugo’s multilingual mode has worked quite well for me so far, but it also means that I maintain separate content and taxonomies for both language versions.

Attentive readers may have noticed me posting links to my website with two separate domains:

The .com domain is in fact the primary one under which the website is hosted. The .de domain redirects to www.seanfobbe.com/de but can otherwise be used as a drop-in replacement for this subfolder and all deeplinks will be honored.

Switching between languages is also possible via the small flag icon in the top right of the website.

I try to keep both language versions as much in sync as possible, but there are exceptions:

Sometimes it takes me a while to translate content.
Sometimes there is very long content or strongly language-dependent content that I feel should be read in the original language; here I will link to the other language version.
Sometimes there is content, esp. German blog posts, that is quite probably not of interest to an international audience that does not speak German; here I neither link nor translate.

If you speak both English and German and feel adventurous, try toggling between the language versions occasionally.

Tags Link to heading

All tags, including the number of posts/pages for each tag, are listed here:

https://seanfobbe.com/tags/

For tags I have come up with the following strategy:

Tags describe thematic content
Each blog post or page should have one or more tags, but no more than 10
Blog posts must have tags, pages may have tags
Tags should generalize to multiple posts, but having a large number of tags to describe thematic content is fine

During the past few days I went over the tags on a number of posts and did some cleaning, but there is still some work to be done. Expect existing tags to change for a while.

You can access thematic content by either clicking the link to the full list of tags above or any of the tags on a post, which will show similarly tagged posts.

Categories Link to heading

All categories, including the number of posts/pages for each category, are listed here:

https://seanfobbe.com/categories/

For categories I have decided on the following strategy:

Categories describe the type of post/page
Each blog post or page can be assigned a maximum of one (1) category¹
Blog posts must have categories, pages may have categories
Categories must generalize to multiple posts/pages and the number of categories should be kept small and tightly controlled

Currently I maintain the following categories:

Announcement concerns future events (e.g. upcoming events or publications) and should always be associated with a date.
Data collects all posts about my open data work and possible information on other data publications that may be of interest to readers.
Essay describes long-form content. Essays are deeper reflections on certain topics, will usually contain a fair amount of opinion, sometimes math, code or diagrams and may be as long as I feel is warranted.
Media is for past interviews, podcasts and other media appearances.
Publications posts inform about new publications such as peer-reviewed papers, reports, articles and similar.
Speaking is for follow-ups on academic speaking engagements, particularly for slides or recordings of talks. This category is more retrospective, upcoming events are categorized under Announcement
Tutorial is for posts and pages with a teaching purpose.
Website is for posts like this one, which inform about features or reflect on the state of this website.

This is a conscious website design decision and not required by Hugo or the Coder theme. In fact, you can have as many categories as you like. ↩︎

The Top 10 Most Structurally Complex German Federal Laws

Thu, 04 Jan 2024 00:00:00 +0000

Overview Link to heading

In December 2023 I created a rough list of the top 10 most structurally complex German federal laws and posted them to social media. The diagrams were drawn from the Corpus des Deutschen Bundesrechts (C-DBR). For this article I’ve written down the list and collected all diagrams into ordered collages.

The original diagrams are available as high-resolution vector graphics from this link: Fobbe, S. (2023). Corpus des Deutschen Bundesrechts (C-DBR) (2023-10-03) [Data set]. Zenodo.

All diagrams are in the public domain and can be used for any and all purposes. Especially for research and teaching!

Want to create better diagrams than I did? I wrote a tutorial for RECHTS|EMPIRE (in German only) on how to do this: Juristische Netzwerkdaten für Einsteiger:innen. Although to be fair, you only need to grab the GraphML files from the data set, import them into Gephi and off you go.

Structural Complexity of German Federal Laws Link to heading

What is the structural complexity of a law? Before I say anything, let me mention that my colleague Dr. Corinna Coupette is the true authority in this field and anyone who is interested in this line of research should check the list of publications on her personal website. Most of her papers are in English, although her main book on legal network analysis is available only in German: Juristische Netzwerkforschung — Modellierung, Quantifizierung und Visualisierung relationaler Daten im Recht (Mohr Siebeck 2019)

My definition of complexity for this list was a lot more relaxed: I listed all PDF files of the dendograms (headings only, that is, excluding individual norms) of all German federal laws in order of their file sizes. The PDF diagrams are vector graphics and their file size increases with the number of nodes and edges, but also the longer the labels are. For a first approximation this turned out to be a surprisingly good heuristic.¹

The Top 10 Most Structurally Complex German Federal Laws Link to heading

The list is led by civil law, in particular the Civil Code and the Commercial Code, following by three books of social law. Positions 5 through 10 are held by laws from different subject areas, but civil law is again notably strong. I give rough translations in [brackets], followed by the original German names below.

[Civil Code] — Bürgerliches Gesetzbuch (BGB)
[Commercial Code] — Handelsgesetzbuch (HGB)
[Sixth Social Code] — Sechstes Sozialgesetzbuch (SGB-VI)
[Fifth Social Code] — Fünftes Sozialgesetzbuch (SGB-V)
[Third Social Code] — Drittes Sozialgesetzbuch (SGB-III)
[Civil Procedure Code] — Zivilprozesordnung (ZPO)
[Tax Code] — Abgabenordnung (AO)
[Corporate Transformation Law]² — Umwandlungsgesetz (UmwG)
[Capital Investment Code] — Kapitalanlagegesetzbuch (KAGB)
[Farmer Insurance Law] — Gesetz über die Alterssicherung der Landwirte (ALG)

Ordered Collages of all Diagrams Link to heading

The collages are ordered from top left to bottom right. Top 1 and top 2 are the top left and top right, respectively. Then top 3 and top 4 are second row left and right. And so on.

Dendrograms Link to heading

Sunburst Link to heading

Circlepacking Link to heading

This is fine for social media, but you probabily will never make peer review with this ;-) ↩︎
This refers to the transformation of corporate entities into different corporate entities. ↩︎

UN Human Rights Council Workshop on Cultural Rights and the Protection of Cultural Heritage

Mon, 04 Dec 2023 00:00:00 +0000

Info

[Open Access] Materials related to the Third Intersessional Workshop on Cultural Rights and the Protection of Cultural Heritage

Intersessional Workshop on Cultural Rights and the Protection of Cultural Heritage Link to heading

On 1 December 2023 the UN Office of the High Comissioner for Human Rights (OHCHR) convened a one-day Intersessional Workshop on Cultural Rights and the Protection of Cultural Heritage, as requested by the UN Human Rights Council (UNHRC) in resolution 49/7.

The UNHRC and OHCHR regularly convene workshops to better study human rights issues and clarify questions of international law. This was the third intersessional workshop on cultural rights and the protection of cultural heritage. The first was held in 2017 and the second in 2021.

I am grateful to the United Nations for their kind invitation to speak at the event, as well as for their significant efforts in organizing the workshop. The hybrid format was well-executed and attracted a diverse and interesting range of speakers from all around the world.

Opening Remarks of UN Special Rapporteur Xanthaki Link to heading

During her opening statement the UN Special Rapporteur in the field of cultural rights, Professor Alexandra Xanthaki, raised a number of important points that deserve attention. In particular:

The international community often focuses too much on cultural heritage per se, instead of on individuals, communities and their right to participate in cultural life
What do we mean by “cultural heritage”? Most States focus on tangible cultural heritage and push aside living heritage and natural heritage; where there are such initiatives, they are not framed in such terms
Cultural rights are for humans, not for States
Cultural rights are granted to all inhabitants, not just citizens
There is not just one cultural heritage, but many cultural heritages (the plural is important!)
Who represents cultural heritage? Even among minorities, indigenous peoples and non-State communities the louder voices attempt to dominate the discourse; we need an effective right to participation for all
Women are often called the “carers of culture”, but are pushed aside when cultural heritage is defined and managed
The folklorization of indigenous peoples and minorities is a problem; in particular, what are the effects of tourism on their cultures?
The World Bank, International Monetary Fund (IMF), the UN Development Program (UNDP) and the World Intellectual Property Organization (WIPO) have not even begun a cultural rights discussion
The redress for violations of cultural heritage has not been addressed much

Remarks by the Head of the Cultural Mission of Timbuktu (Mali) Link to heading

The entire meeting is worth re-watching, but one other contribution I would like to highlight was the testimony provided by Mr. El-Boukhari Ben Essayouti, the Head of the Cultural Mission of Timbuktu, Mali.

Mr. Ben Essayouti provided first-hand insights on the time when Timbuktu was held by Islamists and much of the destruction of mosques and mausolea occurred that was later the subject of the criminal proceedings against Ahmed Al Faqi Al Mahdi before the International Criminal Court (ICC). In 2016 Al Mahdi was found guilty of directing attacks against nine mausolea and one mosque in Timbuktu during the 2012 occupation.

I think the experiences of Mr. Ben Essayouti are best listened to directly. I’ll post the link to the recording as soon as it is available. His written statement is already online.

My Statement during the Workshop Link to heading

Later, during the discussion part of Panel II, the Chair of the meeting kindly granted me the floor to intervene on behalf of RASHID International, an international NGO in special consultative status with the United Nations since 2019. I raised two points that deserve the attention of the international community:

Advanced chemical tracer technology to protect cultural objects
Lack of attention to the List of Cutural Property under Enhanced Protection

The statement reflects intense work carried out by the international non-profit organization RASHID International over the years in the field of cultural heritage and cultural rights. The full statement is reproduced below.

PANEL II: Cultural Heritage in Crisis and Work of Cultural Rights Defenders (15.00–17.00 CET)

Distinguished Delegates, Ladies and Gentlemen,

my name is Seán Fobbe and I serve as the Chief Legal Officer of RASHID International, an international non-profit organization dedicated to protecting the cultural heritage of Iraq. We have been in consultative status with the Economic and Social Council of the United Nations since 2019.

A core recommendation to protect cultural heritage during crisis is the advance preparation of inventories. We urge all actors to think beyond standard procedures such as written descriptions, drawings and photographs by also considering more advanced technology. Our own work in Iraq, led by Professor Roger Matthews, has focused on applying invisible chemical tracer liquids with encoded signatures to inorganic artefacts, such as stone, metal, glass and clay. This security technology was applied to more than half a million artefacts in five Iraqi museums, permanently protecting them from theft as it clearly documents provenance and allows recovery and prosecution decades down the line. Furthermore, it is impervious to forgery of provenance, as well as loss of records at the relevant site.

Legal measures are another important part of the mosaic of international heritage protection. We are concerned that the List of Cultural Property under Enhanced Protection remains almost unused. The List was created under the 1999 Second Protocol to the 1954 Hague Convention in order to protect heritage of the greatest importance for humanity. It currently lists only 18 sites under full Enhanced Protection and 20 sites under provisional Enhanced Protection. Compare this to the approximately 1,200 sites on the World Heritage List. We urge the international community to not let this instrument fall by the wayside and to ensure that more sites are given Enhanced Protection.

Thank you.

100,000+ Legal Dataset Downloads

Sun, 03 Dec 2023 00:00:00 +0000

Info

Open Data Repository — data.seanfobbe.com

Thank you! Link to heading

Thank you all for for your interest in my work! My Open Access repository of legal datasets over at Zenodo passed the milestone of 100,000 downloads sometime during November. I am ecstatic that something appears to be changing in the legal world and interest in Legal Data Science is increasing.

That being said, as you can see from the diagram below, the distribution of attention has been rather asymmetric and skews towards a few especially popular datasets. There are plenty of interesting applications to be worked on with some of the lesser known data sets, so I encourage you to browse around a bit and see if you find anything interesting.

Some top recommendations:

The corpus of German parliamentary materials (CDRS-BT) is vast (more than 800 million tokens) and contains draft laws, in addition to other materials, dating back to 1949
The set of German legal stopwords (SW-DE-RS) is an auxiliary data set that can save you a huge amount of work when cleaning your own German legal data
The Collection of International Treaties and Legal Documents (CITLD) is an excellent companion to traditional international law research — myself, I use it almost daily

Applications Link to heading

If you happen to have built and published any interesting applications or research with these datasets, write me and let me know!

Zenodo Download Statistics Link to heading

[Essay] Modeling Data Workflows with {targets}

Sun, 12 Nov 2023 00:00:00 +0000

Overview Link to heading

In this article I discuss my experience modeling data pipelines with the R package {targets} (Landau 2023) and some of the principles I follow in constructing them.

{targets} pipelines are a significant departure from the monolithic linear data workflows that are common in data science and which I myself have used heavily in the past (you can still see one at work in the ICJ corpus). Declarative workflows break up the pipeline into different components that are run in an order determined by the {targets} framework and the interlinked dependencies of the individual components.

This has many benefits. In particular, this type of pipeline significantly decreases development time, is easier to fix when broken and allows for shorter development increments. It is possible to spend an hour here and there instead of having to reserve an entire afternoon while working through a linear pipeline.

The downside is a fair amount of additional work when setting up the pipeline and the added intellectual complexity of the {targets} framework, but on the whole I am very satisfied will never go back to linear workflows (except for minor analyses).

I won’t claim that the principles for modeling pipelines presented in this article are best practice or comprehensive, but there are nevertheless some finer points that colleagues may appreciate.

Table of Contents Link to heading

Data Workflow Link to heading

If you follow my work you may have seen me post striking neon-on-black network visualizations of my data pipelines. I use these for three purposes:

Check the structure of my computations
Discuss the details of my work with a technical audience
Explain the general outline of data workflows to a non-technical audience

The most common visual I use to display pipelines is a network representation of the data workflow, also called a directed acyclic graph (DAG). Data workflow means that the diagram below includes only targets, the primary data components of the pipeline. Excluded are functions (i.e. computational instructions) and global variables.

Info

The workflow diagrams shown in this article represent a pipeline that connects to the official source of German federal legislation and constructs a complete and cleaned data set of all German federal laws and regulations, the Corpus des Deutschen Bundesrechts (C-DBR) (Zenodo 2023). Note that the documentation is in German.

Note

Network Viz Algorithms

I have tried every algorithm under the sun — or at least as contained in {ggraph} — to visualize data pipelines and have found that the only one that works well for data workflows is sugiyama. I encourage you to try different options yourself, but always remember sugiyama as a good option. At the end of the article you will find R code to produce these diagrams based on this algorithm.

Each target is a self-contained part of the pipeline that is computed in its own R process, with a specified reproducible seed and with its output stored as an individual R object, which can be accessed individually with tar_load() or tar_read(). Every target is defined by three elements:

Data input
Code
Data output

The small white dots in this diagram show the output and the link to the input(s), but not the functions. Target outputs usually are vectors or data frames (in my case almost always data.tables). Sometimes these vectors reference files on disk (e.g. when a target downloads thousands of PDF files). The framework computes hashes of all stored results (including references files on disk) and tracks changes to them. When the pipeline runs it automatically skips over targets that have already been computed and for which input and output remain unchanged. This feature alone makes it worthwhile to build {targets} pipelines.

Note

Targets Storage Format

The default storage format for targets is Rds. You can easily change this to the qs format, which is faster to read/write and uses less disk space due to better compression.

Simply add tar_option_set(format = "qs") to your global targets settings and make sure the qs package is installed in your package library.

I almost always use a targets-only visual of the workflow. Why?

The full structure (including functions) and globals becomes cluttered very quickly even with moderately-sized pipelines
A data pipeline is about the flow of the data, everything else is incidental to it
I find it easier to anchor my inspection of the code with data targets and then look at the associated function in the pipeline manifest .Rmd if there is an issue
It is visually more attractive

When I review the pipeline diagram, I always ask myself the same questions:

Are there dangling targets?
Are all targets connected to the proper upstream/downstream targets?
Does the overall pipeline concept make sense?

Dangling targets tend to happen fairly often during development, particularly when I’ve written all the necessary functional code but have forgotten to add downstream targets to the pipeline. Visual inspection shows this quickly. Sometimes I also comment out parts of the pipeline because I am working on upstream parts where frequent changes would cause the downstream part to rerun often. Visual inspection reminds me to uncomment these parts when I am done with upstream development work.

Checking proper connections is more tricky, but there are some elements of my own modeling style that are prone to improper links, so I re-check them often. For example, I tend to build a preliminary metadata download manifest from whatever database or file set I am currently extracting data from, but this often requires additional cleaning before I run the full data content extraction. Sometimes I continue to work on the pipeline before I have done the cleaning, so I hook up the preliminary data to downstream targets. When I add the cleaned download manifest to the pipeline I always check if I’ve severed the downstream connections with the preliminary data and added the final download manifest as a data source.

Most importantly, the overall workflow needs to make sense. If the pipeline structure is all over the place and doesn’t seem very intelligible I try to redesign the pipeline to make the process clearer, group similar targets together in the manifest and workflow and so on. Mistakes in programming happen more frequently if structure is poor, one of the reasons why re-use of functions and use of control flow is encouraged instead of burying problems under mountains of code.

Note

Literate Programming with {targets}

With complex pipelines it becomes cumbersome to write the _targets.R file by hand. {targets} offers a convenient style of literate programming at the pipeline level, which enables construction of the pipeline in an .Rmd file. This is what I always use. See the literate programming section of the {targets} manual for details.

Modeling Targets Link to heading

The construction of individual targets is more of an art than a science. At its core, each target can be thought of as a building block or component of the full workflow. The {targets} package forces you to use a single R expression (commonly a custom function) to create each target.¹

The question is then, how much should you fit into each target and its associated function? There are several criteria I use to decide:

What is the runtime of the target?
Can I avoid re-running computations by creating new targets?
How often will I be refactoring/fixing the target?
How interested am I in the results for a specific step of the workflow?
Can I collect several related computations into a conceptual unit?
Does it improve intelligibility in the context of the entire pipeline?
Can I reuse the function that builds a target across different projects?

Runtime and Re-Run Avoidance Link to heading

Runtime means taking advantage of the fact that {targets} skips steps that have have already been computed for which the input data, code and output file(s) remain unchanged. For example, if I have a component that a) runs for 60 min, b) could be easily split into two parts and c) keeps producing errors in the second 30 min, it can make a lot of sense to split it into two targets so I don’t have to rerun the first 30 min all the time. This is one of the major reasons why I love {targets} so much. This happens a lot in my line of work.

Code Changes Link to heading

Runtime ties directly into my next point: how often do I need to change the code or input data that goes into a target? Some components of some pipelines just get continuous or regular development attention, for example if they frequently produce downstream problems, are affected by input data changes (particular if the target fixes data coming from a source I don’t control) or contain features that I have not fully developed yet. Partially developed features especially are good candidates for separate targets that branch out from the main pipeline.

Pipeline Inspection Link to heading

Some targets I am especially interested in and would like to review their results often. This can be as part of a formal test in a literate programming report or as informal testing if I suspect something will often go wrong at a certain step.

This may also be a concern if a target contains many strict data quality checks that can cause it to error out if the computational result is questionable. I build scientific pipelines that tend to err on the side of perfection rather than aiming for high uptime, so this may or may not be as relevant to you.

Conceptual Units and Pipeline Intellibility Link to heading

Some targets have low runtime, don’t change often and don’t need much inspection, but I nonetheless model them separately to have sensible conceptual units from which I can continue to build the pipeline and which make the entire workflow more intelligible. This comes up in particular when I am extracting new variables from existing variables (e.g. via REGEX from a string variable). I like to have each variable or group of variables as a conceptual unit before I recombine and merge them with the data they were extracted from.

Function Reuse Link to heading

An important concern for those who work on multiple projects is function reuse. When you have a custom function with generic applicability that is useful across many projects it makes sense to always model it as an individual target. My classic use cases are my custom download function f.download (it has a silly amount of features) and the conversion of PDF to TXT files. These are components that I reuse in almost all projects and always model as their own target(s).

Parallelism Link to heading

Parallelism also factors into these considerations. Every data pipeline I write includes some form of parallel computation to reduce runtime. Parallelism can be an intimidating subject at first, but even if you have never used it before, I recommend reading an introduction as soon as you reach intermediate level as an R programmer. In terms of pipeline construction, parallelism determines the type and number of targets you are likely to create, so familiarity with the basics is important.

You might encounter parallelism in R at four different levels:

The {targets} framework permits processing multiple targets in parallel based on {future}
Explicit parallelism when you use functions like mclapply or the {future} framework
Implict parallelism when package authors have added parallel processing to their functions
Parallelization at the system level, such as with the basic linear algebra subsystem (BLAS)

Watch out for implicit parallelism, as this may throw off your allocation of system resources if you overlook it. I haven’t encountered parallelism at the system level very often. The {stm} package in combination with a compiled BLAS is the only surprise I’ve encountered so far. If system parallelization is used it is probably part of an R package design decision and documented as such.

Most often you’ll encounter some kind of implicit parallelism as part of R packages. Often there will be an option in a function like cores = parallel::detectCores() that sets the number of cores to maximum by default. In a linear workflow this is not an issue, but if you add your own explicit parallel framework or process targets in parallel this may become a concern through overloading your machine’s resources.

For pipeline design the important considerations are a) explicit parallelism and b) parallel target processing. If you have a single machine, the former is likely to be your method of choice, if you have multiple machines or nodes, a combination of the two.

In my workflows I usually encounter a few targets that must process a heavy workload, but are strict upstream dependencies for the rest of the workflow (bottlenecks). I tend to model these bottlenecks with explicit parallelism contained in a single target, since creating additional targets for each would cause too much computational overhead and negatively impact the legibility of the workflow diagram (in case of manual or static branching, not applicable to dynamic branching). If your workflow has fewer or no bottlenecks (i.e. is conceptually more parallel when viewed in the diagram) it may make more sense to run several targets (or parallel lines of targets) in parallel.

Note

Parallel Frameworks in R

There are quite the number of parallel frameworks in R, so I am just going to mention my favorites. If you are already using higher-order functions from base R like lapply() or sapply in your workflow, it is easiest to simply use companion functions prefixed with ‘mc’ such as mclapply() or mcsapply and you are instantly running everything in parallel.

For those (like me) who have more advanced needs, the {future} framework is your best bet. The {future} framework is more than just a package, it allows you to select a custom frontend and backend to adapt the system to your personal needs. For example, I use {future.apply} as a frontend, which provides functions that behave almost like the apply() family in base R. If you happen to prefer the foreach frontend, you can use this with {doFuture}. There is also the {furrr} package which interfaces with the tidyverse. I usually use the standard parallel backend, but sometimes also callr, which is the parallel backend that {targets} uses for parallel processing of targets.

Complete Pipeline (Data, Functions and Globals) Link to heading

A visual of the complete pipeline can be quite intimidating. It displays a harsh beauty, but there is usually too much going on to make much sense of it. Even dynamic tools like the function tar_visnetwork() have trouble making sense of it all. I would recommend exporting the graph to a .graphml file and loading it into Gephi, which is a more convenient tool to view graph diagrams at this level of complexity.

Nonetheless, adding functions and globals to the diagram can provide us with some important clues:

Are all functions hooked up to the right targets?
Are there dangling functions that are not used in the pipeline?

As you can see, there is a dangling set of three function nodes on the right of this diagram. These are part of an older PDF to TXT extraction workflow that I will probably clean up at a future point in time, but are still part of the project in case I want to refer to them.

Reproducing Data Workflow Diagram Link to heading

To reproduce the above diagram, run the C-DBR pipeline at least once (the pipeline is stored in .Rmd and needs to be rendered to .R first), then execute the below code in the same working directory as the project. You can quickly get the project by git cloning https://github.com/SeanFobbe/c-dbr

The dimensions for the static image are 20x20 inches at 150 dpi as PNG.

Note that every publication of the C-DBR corpus alwys contains a static snapshot visual of the pipeline as PDF/PNG in the analysis archive.

 1library(targets)
 2library(data.table)
 3library(igraph)
 4library(ggraph)
 5
 6edgelist <- tar_network(targets_only = TRUE)$edges
 7setDT(edgelist)
 8
 9g  <- igraph::graph.data.frame(edgelist,
10                               directed = TRUE)
11
12
13ggraph(g,
14       'sugiyama') + 
15    geom_edge_diagonal(colour = "darkgoldenrod2")+
16    geom_node_point(size = 2,
17                    color = "white")+
18    geom_node_text(aes(label = name),
19                   color = "white",
20                   size = 2,
21                   repel = TRUE)+
22    theme_void()+
23    labs(
24        title = paste("C-DBR",
25                      "| Complete dataset creation process (data only)"),
26        caption = "Fobbe | DOI: 10.5281/zenodo.3832111"
27    )+
28    theme(
29        plot.title = element_text(size = 14,
30                                  face = "bold",
31                                  color = "white"),                        
32        plot.background = element_rect(fill = "black"),
33        plot.caption = element_text(color = "white"),
34        plot.margin = margin(10, 20, 10, 10)
35    )

Reproduce Complete Pipeline Diagram (Data, Functions and Globals) Link to heading

The complete pipeline diagram is almost the same as the data workflow above, except that we set tar_network(targets_only = FALSE) and the plot title is slightly different.

The dimensions for the static image are 20x20 inches at 150 dpi as PNG.

Note that every publication of the C-DBR corpus alwys contains a static snapshot visual of the pipeline as PDF/PNG in the analysis archive.

 1library(targets)
 2library(data.table)
 3library(igraph)
 4library(ggraph)
 5
 6
 7edgelist <- tar_network(targets_only = FALSE)$edges
 8setDT(edgelist)
 9
10g  <- igraph::graph.data.frame(edgelist,
11                               directed = TRUE)
12
13ggraph(g,
14       'sugiyama') + 
15    geom_edge_diagonal(colour = "darkgoldenrod2")+
16    geom_node_point(size = 2,
17                    color = "white")+
18    geom_node_text(aes(label = name),
19                   color = "white",
20                   size = 2,
21                   repel = TRUE)+
22    theme_void()+
23    labs(
24        title = paste("C-DBR",
25                      "| Complete dataset creation process (with functions and globals)"),
26        caption = "Fobbe | DOI: 10.5281/zenodo.3832111"
27    )+
28    theme(
29        plot.title = element_text(size = 14,
30                                  face = "bold",
31                                  color = "white"),                        
32        plot.background = element_rect(fill = "black"),
33        plot.caption = element_text(color = "white"),
34        plot.margin = margin(10, 20, 10, 10)
35    )

A Word of Thanks Link to heading

My special thanks go to William Michael Landau and the {targets} contributors for creating such an amazing piece of software. Next to {data.table} and {future} this is one of my all-time favorites of the R ecosystem.

The exception to this are target factories, that is, functions that create multiple targets at once. Refer to the sections on static and dynamic branching in the {targets} manual and have a look at some of the functions in the {tarchetypes} package. ↩︎

This is a Zero Tracking Website

Thu, 19 Oct 2023 00:00:00 +0000

I have zero trackers or cookies on my personal/academic website. Not even open-source variants.

Why?

When I set up the website I realized I really, really don’t care who reads what when for how long
I also realized I am too lazy to set up extra software
Cookie banners are annoying as hell
Site loads faster
Can’t compromise data you don’t have

If you find anything on https://seanfobbe.com/ that contradicts this (and is under my control), please let me know and I’ll take care of it.

[New Dataset] German Federal Court of Finance Corpus (CE-BFH)

Tue, 17 Oct 2023 00:00:00 +0000

Info

[Open Access] Corpus der Entscheidungen des Bundesfinanzhofs (CE-BFH) (Zenodo 2023)
[Open Access] Source Code des ‘Corpus der Entscheidungen des Bundesfinanzhofs’ (CE-BFH-Source) (Zenodo 2023)

Overview Link to heading

Following a long period of consolidating my previous work I am happy to announce a new dataset release!

The Corpus der Entscheidungen des Bundesfinanzhofs (CE-BFH) is a complete collection of all decisions published by the German Federal Court of Finance (Bundesfinanzhof or just BFH). The dataset connects to the official database of the Bundesfinanzhof and transforms it into a machine-readable dataset.

Please make sure to read the accompanying Codebook! The Codebook is essentially the user manual for the dataset. It contains important information if you wish to use the dataset correctly and should be the first port of call for any new and experienced user. It will also help you decide which variant is right for you. Usually I recommend the CSV variants for quantitative applications and the PDF variant for traditional research.

For practitioners there is an additional variant that contains only the decisions published in the official collection (BFHE), also known as V-Entscheidungen.

The dataset will likely be updated once or twice per year. I always post recent news about new and updated datasets to Mastodon at @seanfobbe@fediscience.org.

Note

Please note that the dataset documentation and variables are in German. One reason for this is the loss of information when translating something from German to English and back again. Another is that anyone conducting NLP on German-language documents should speak either enough German to be able to read the documentation or probably doesn’t care about the details anyway since everything is fed into an LLM without much consideration for data quality.

Workflow Diagram Link to heading

Key Facts Link to heading

Reference date: 15 October 2023
Scope: 10,310 Decisions of the Federal Court of Finance (Bundesfinanzhof) of the Federal Republic of Germany
Formats: CSV, PDF, TXT and HTML

Features Link to heading

34 variables
Regular updates
Public Domain (CC-Zero 1.0)
Open and platform independent file formats (PDF, TXT, CSV, HTML)
Extensive Codebook
Compilation Report explains construction and validation of the data set in detail
Large number of diagrams for all purposes (see the ‘ANALYSIS’ archive)
Diagrams are available as PDF (for printing) and PNG (for web display), tables are available as CSV for easy readability by humans and machines
Secure cryptographic signatures
Publication of full source code (Open Source)

No Copyright: Public Domain Link to heading

The texts and metadata are in the public domain (§ 5 para 1 UrhG) as they are court decisions and official works. § 5 para 1 UrhG is also applicable to official databases via analogy, as the German Federal Court of Justice has decided (BGH, Beschluss vom 28.09.2006 - I ZR 261/03, “Sächsischer Ausschreibungsdienst”).

All of my own contributions are released into the public domain through the CC0 1.0 Universal Public Domain License.

Disclaimer Link to heading

This dataset is a private academic initiative and is in no way associated with agencies, courts or other public authorities of the Federal Republic of Germany.

Open Source Tools List Updated and Expanded

Wed, 11 Oct 2023 00:00:00 +0000

Curious about my tech tools and workflow? I’ve updated and greatly expanded the annotated list of favorite open source tools on my website: https://seanfobbe.com/code/

Includes both general tools (browser, e-mail, operating system) and data science specific tools (R, R packages, reproducible science).

German translation coming soon!

[Essay] Some Thoughts on the Use of Large Language Models in the Legal Domain

Thu, 27 Jul 2023 00:00:00 +0000

Info

Recommended Reading

Climbing towards NLU (Bender and Koller 2020)
On the Dangers of Stochastic Parrots (Bender et al 2021)
Situating Search (Shah and Bender 2022)
Mystery AI Hype Theater, Episode 10 - Don’t Be A Lawyer, ChatGPT (DAIR)

Fake Filings in Federal Court Link to heading

’tis the season for AI unreason once again.

This post was prompted by the recent-ish (but inevitable) news item in which an attorney used ChatGPT to generate a legal brief (including fake citations and fake case law) for another attorney to file in US Federal Court on behalf of his client (Mata v. Avianca, Inc.). And, of course, it went very, very wrong.

Yes, we are at stage of the hype cycle where the output of Large Language Models (LLM) is prompting humans, not the other way around.

The full case history is available on CourtListener. The Guardian offers a brief review. I just want to highlight a few notable nuggets:

The apology is particularly interesting, esp. as a cautionary tale
The attorney did not just submit fake citations, but also fake excerpts and notarized them

In the end the attorneys and their firm were fined USD 5,000. Judge P. Kevin Castel had the following to say:

In researching and drafting court submissions, good lawyers appropriately obtain assistance from junior lawyers, law students, contract lawyers, legal encyclopedias and databases such as Westlaw and LexisNexis. Technological advances are commonplace and there is nothing inherently improper about using a reliable artificial intelligence tool for assistance. But existing rules impose a gatekeeping role on attorneys to ensure the accuracy of their filings. [The attorneys and law firm] abandoned their responsibilities when they submitted non-existent judicial opinions with fake quotes and citations created by the artificial intelligence tool ChatGPT, then continued to stand by the fake opinions after judicial orders called their existence into question.

Mata v Avianca, Opinion and Order on Sanctions, 22 June 2023

In other words: check your sources and don’t lie to the judge.

The “ChatGPT Lawyer” is likely to end up becoming part of modern tech history, much as the “Zoom Cat Lawyer” did. Although the latter is far more wholesome and one of my favorite pieces of legal internet trivia.

Table of Contents Link to heading

Side Note on Citations Link to heading

The problem of fake legal citations is fixable at the technology level, but simply wasn’t a concern for the ChatGPT developers. Since LLMs operate at the word level (more often called a “token”), one would only need to pre-process the training data to concatenate legal citations into single tokens. A citation string like “1 BvR 141/16” would be converted into 1_BvR_141/16" or similar.

As far as we know, LLM hallucinations are not due to the creation of entirely new tokens, but due to the senseless or incorrect arrangement of known tokens from the training data. So a specialized legal LLM would not necesarily suffer from incorrect citations, but the fake quotes are more difficult to fix.

One would, of course, need to invest a fair amount of money to gather all relevant citation patterns. Presumably many legal publishers have already done so years ago. Even for new entrants into the market, given the volume of announced investments, this seems like a feasible effort.

Hype and Hallucination Link to heading

Following the recent hype arounds LLMs it is unsurprising that all sorts of actors are under pressure to invest in and deliver LLMs for their customers and stakeholders. OpenAI led the charge with ChatGPT, GPT-3 and GPT-4, Microsoft immediately partnered with OpenAI to bring Bing out of its twilight exile and Google was forced to prematurely announce Bard in a desperate bid to maintain control of the search market. A number of other organizations have announced billion dollar investments into generative AI.

The legal profession has been anything but unaffected by the hype, with many firms trying to apply one of the GPT variants to legal problems, some companies offering custom legal models of unknown quality or developing entirely new offerings. Even the famously tech-averse, risk-averse and conservative German judiciary has announced the development of a custom language model for judicial use.

But is this a reasonable response? People who stand to earn a lot of money from generative AI certainly think so. Already there are claims that the “The AI Revolution Will Be More Significant Than the Agricultural Revolution” (Sam Altman, CEO of OpenAI) or that “AI will save the world” (Marc Andreessen).

If I were hoping to sell a billion subscriptions that’s what I’d be telling my customers and investors as well. In his article Andreessen lambasts bootleggers, meaning “self-interested opportunists who stand to financially profit by the imposition of new restrictions, regulations, and laws”. He conveniently forgets to mention snake oil salesmen — people who profit immensely from selling the “next big thing”, regardless of whether it is useful or not.

On a technical level, LLMs currently rely on “Transformer” technology. Transformers “learn” about text by being fed sequences of words, with the final word masked. The model then has to guess the masked word. Stephen Wolfram has a more detailed and excellent write-up of the details.

So the sentence “It is illegal to commit murder” is fed to the algorithm as “It is illegal to commit [???]”. After being fed incredible amounts of data, the model might learn that “murder” is 30% probably, “theft” is 20% probable and “crimes” is 10% probable. With a larger context window than this short sentence the guesses become better suited to the prompt.

There is also a “temperature” parameter, which defines a certain probability to randomly select a lower-ranked word, because this makes texts “more interesting”. Essentially this means two levels of probability: the probabilities of the next likely words mixed with the probability of a lower-ranked word being selected.

On a more philosophical level, much has been made of the emergent abilities of LLMs, which at certain scales appear to suddenly exhibit abilities like natural language understanding, logical reasoning or even conscience. But do they? Schaeffer, Miranda and Koyejo (2023 / pre-print) call this view into question, arguing that “emergent” abilities are simply a result of nonlinear/discontinuous metrics and completely disappear if you change the evaluation method.

In brief, Transformers try to guess the next word based on the input prompt, given the probabilities it has learned from large amounts of text. This also means that Transformers create plausible text, but at the fundamental technical level do not understand it (Bender et al 2021), whatever “understanding” may mean in this context. Nor does the underlying technology provide them with any reasoning capabilities, except by accidental and plausible regurgitation of training data (Shah and Bender 2022: 222).

Where does that leave us? There are plenty of theoretical reasons to suppose that LLMs are unlikely to be some kind of super-intelligence, but instead are more akin to “stochastic parrots” (Bender et al 2021) or “spicy autocomplete”.¹

I am not an expert in LLMs so I cannot speak to the technical and mathematical quality of the critiques, but it seems rather likely that a huge string prediction engine trained on (allegedly) mostly internet text would turn out to be nothing more than a huge string prediction engine producing remixed internet text.

Large Language Models in the Legal Domain Link to heading

The legal profession has never had access to one or more all-powerful, all-knowing AIs before (if they are that), nor to fancy string-prediction enginees (my guess). And yet, the work is still getting done. So what would you use LLMs for?

Some tasks one could apply LLMs to include ideation, legal research, legal drafting, summarization and programming for lawyers. Naturally, there are many more potential applications. This list is certainly not comprehensive, but rather an opportunity to structure my own thinking on this topic.

Ideating Link to heading

Ideating seems like an ideal use case for LLMs. You throw a prompt at your favorite chatbot, the chatbot replies with a couple of ideas, you pick your favorite and voilà, your fantastic new project is out of the brainstorming/-writing phase and on its way to becoming reality.

At this stage of the process, truth, safety and copyright aren’t an issue. Whether something actually works isn’t a concern during brainstorming, as long as it plausibly could and can make the shortlist for further investigation. Safety is also not (yet) engaged. Everything occurs under the purview of a human and can only become reality through human action. Finally, ideas aren’t copyrightable in general, no matter where the idea is taken from.

If academic norms apply to you, you might get in trouble for plagiarism, but this a professional code of ethics that is enforced with professional censure, not legal sanctions (usually).

I assume this might be viable use case, depending on your needs? I’ve tried ideating with some of the publicly available chatbots and in my (very limited!) experience it was rather…disappointing. The results were sort of acceptable, but if you’ve spent any amount of time ruminating on your problem the results weren’t any kind of game-changer.

This is probably one of the real limitiations of ideating with LLMs: the suggestions are derivative of whatever exists in its training data. However, we need creative thinking most when we have new problems that need new solutions. Of course human creativity is also derivate of existing cultural patterns, but humans have access to their own lived experience to add a very recent and unique twist to their thoughts, something which is not represented well in textual data alone.

Maybe LLM ideating will be useful? For some people? I’m not sure, but if you find that the more pedestrian creativity hacks like talking a walk, making a cup of tea, talking to people or having some fun don’t work for you I suppose it could become a tool in your arsenal.

Legal Research Link to heading

Legal research is arguably the hardest problem for an LLM to tackle. The current transformer technology is, at its core, nothing more than a prediction of the next likeliest word given its input context window, with some additional randomness thrown in (“temperature”) to make the text a little spicier. You might get what you are looking for from your prompt, but maybe there is a 80% chance it’ll add some spice to your results to make them more fun, but less relevant to you. Recall the ChatGPT lawyer. If you present fake citations and fake direct quotes to an irate judge, spice will not make them play nice.

Legal research — and more generally, information retrieval (IR) — has two fundamental requirements (Shah and Bender 2022):

Truth
Relevance

If you go looking for something, your result must should be both true and relevant to your search to be useful. If it is untrue, but relevant, you have not retrieved anything, you have created something. If it is true, but irrelevant, you may have learned something new about the world, but not solved your problem.

Truth Link to heading

Truth in this sense can be either correspondence (correspond to the “real” world, whatever that may be) or coherence (deriving from an accepted body of knowledge). A true correspondence fact might be a the location of a physical item, a true coherence fact might be the authentic text of a judgment of the European Court of Human Rights, irrespective of whether the content corresponds to reality or not. The kind of truth you are seeking depends on your problem and your associated query.

In the legal domain, the sources of truth for LLMs are both very good and very bad at the same time.

The source of truth are bad, since the training data most likely contains all or most of the text on the internet and if you believe everything on the internet you probably have (a) never been online before or (b) are terribly naive. This internet text also contains some of the most offensive and immoral content on the planet, which can only be imperfectly removed from the training data through automatic means and surfaces every once in a while through clever prompting or sheer accident.

The sources of truth are also very good at the same time, because (modern) law is one of the few domains that possesses one or more authorized corpora of knowledge (laws, regulations, judgments, other government documents) that are both hierarchical and “true” by fiat. You may feel incandescent rage at the holdings and reasoning of certain Supreme Court judgments, but it doesn’t change the fact that they form authentic interpretations of the law of the land and you must either adhere to them or oppose them as such — but you cannot ignore their power.

Now, what happens if you combine bad sources of truth and good sources of truth? It is really anyone’s guess what kind of fuzzy truth value an individual query will produce and I think that is part of the problem.

If the query produces novel text that is supposed to contain the answer to your question, every word, every sentence, is potentially suspect and will require you to verify its truth value manually. If the answer to the query is trivial and you can verify it immediately, you probably never needed to search for it with an LLM in the first place. If confirming the veracity of the result is difficult, you will need to rely on a second search engine or other outside knowledge to confirm the truth value, which is silly and time-consuming.

In most of the interesting cases it will also be hard to decide whether something is true or false in a binary sense, but our willingness to accept something provisionally as truth will depend on the quality of the methods underlying the result. This applies to most empirical research in the natural and social sciences. If you generate a plausible result with a rigorous randomized controlled trial I will be more inclined to believe it than if you probabilistically recombined some input text and claim that your AI god delivered it from heaven in response to your prompting prayer.

Since it is computatonally infeasible or philosophically impossible to compile a complete list of all possible queries with all possible responses, it also not possible to test either the whole universe of queries or even a representative sample, because there is no sampling frame. No matter how much testing one does, it always remains possible to end up in some kind of local minimum were the training data is sparse and truly bizare results occur.

And even if it were possible to appropriately sample and test the model, the complex end-to-end interrelations between everything in such a model mean that everytime the training data changes substantially, one would have to rerun all tests to discover how likely a truthful response might be.

As the ChatGPT lawyer learned, even if the machine appears to be quoting from a judgment and swears to its correctness, you cannot trust it. I am therefore skeptical that this mode of operation will enhance legal research, simply because the cost-benefit ratio of checking the full output will be worse than performing the research some other way. Essentially, you are doing the research twice.

If the query produces links to a document or provides a selection of authorized excerpts of a document from a trusted database and ranks them for you, I could see LLMs serving a role to meet legal research needs. But this raises the question of relevance.

Relevance Link to heading

Finding relevant legal documents is no fun. Or rather, it is no fun with the commercial and non-commercial legal databases I have had access to in my legal life until now. Let us name no names, because hell hath no fury like a database scorned.

If you know exactly what you are looking for, most legal databases have worked fine for me, with some notable exceptions. With a few databases, even a single typo in the docket number can invalidate your query (looking at you, official German federal court databases).

Given that it is 2023, this is awful, but it is how it is. If the databases are publicly indexable, it is often easier to run a Google search and then click the deeplink.

Now, this would appear to provide an excellent opportunity for an LLM to steal the spotlight by providing search that understands what you really want and to provide it to you. Again, if it provides you with a custom text to serve your answer, everything is suspect. But what if it only selects from a finite set of responses from an approved database (e.g. documents or parts of documents)?

Shah and Bender (2022) have commented much more eloquently and expertly on this problem than I could, so I refer you to their paper for a detailed critique. As I understand the paper, search is a complex activity which relies on much information that cannot easily be transmitted to or reasoned over by the executing machine and therefore the machine alone cannot fully replicate the user’s close cooperation, context-bound information needs and regular updating of relevance considerations in response to the machine’s output.

In terms of opacity, the search functionalities of commercial and non-commercial databases are already so opaque that I do not believe that an LLM would add an extra layer of inscrutability, certainly no more than any other algorithms available.

So. Retrieving approved information objects from a database ranked by relevance via LLM might work? I am told it would be computationally more expensive than current approaches, but I am not expert enough to comment on their efficiency. Time will tell, I think.

Personally, I use explicit computer-assisted search functionalities almost exclusively to locate pieces of information that I already know or suspect to exist. Of the three search scenarios mentioned in Shah and Bender (2022: 225), I usually only carry out computer search to accomplish specific tasks. For exploration and learning I find a more social and linked reading search on Connected Papers or Wikipedia to be much more useful, followed by books, review papers, newspaper articles and information nuggets gleaned from social media. This mirrors the search patterns described by Shah and Bender (2022: 223–224)

I’m not sure that I would profit much from LLM search, but I have been wrong before.

Legal Drafting Link to heading

Blank page syndrome is bad, writer’s block is worse. Everyone who has written more than a few paragraphs knows the dreaded nothingness of a white screen on which our lucid thoughts and witty banter should be surfacing in pleasing regularity.². And if you stare too long into the abyss of the editor, the editor stares back at you.

Writer’s block is a serious problem. I have been waiting for more than 12 years for Patrick Rothfuss to finish Doors of Stone. Clients do not have 12 years. Sometimes 12 hours are too long. Perhaps an LLM can speed up this process, at least for the busy lawyer?

It depends. Legal drafting knows roughly three archetypes:

Templating
Modular drafting (boilerplate)
Bespoke drafting

Templates Link to heading

Templates are one of the foundations of the legal world. Templates save time and money, ensure consistent style and reduce legal risk through tried-and-court-tested language. The archetypical template is a complete and internally consistent document designed for a specific purpose (e.g. a non-disclose agreement, but also model laws or instruments of ratification) and only contains some variables such as the names of the parties, addresses and fields for dates and signatures. You create it once and you’re done (the exception being updates due to changes in legal or factual circumstances). Variables can be filled automatically from a database or mindlessly by hand.

The promise of LLMs includes the ability to call up custom templates for any kind of legal situation at a moment’s notice. Unfortunately, producing a perfect, ready-to-use and court-proof template is (currently) out of scope for LLMs. The stochastic nature of the text-generating process means that every word is possibly suspect, so the entire text needs to be double-checked by a legal professional with appropriate expertise.

Whereas standard templates have almost zero marginal labor cost, LLM-generated templates incur a non-trivial cost with each usage, making LLMs unsuitable for on-demand archetypical templates.

Modular Drafting Link to heading

Full templates are quite inflexible and therefore only applicable to a limited class of problems that is easily automated and requires next to no judgment calls. Most legal problems are routine, but do require some measure of customized judgment. Also, lawyers need to keep busy to justify their significant salaries, so “templating and chill” will not satisfy internal and external clients.

Modular drafting is a form of semi-templating. The building blocks (modules) are themselves small templates (boilerplate) and then recombined according to data-driven rules at scale or human judgment in an individual workflow. These building blocks could be single sentences, but also paragraphs, whole sections or even entire documents. Like full templates, these modules may be static or contain variables for fields such as dates, currency, names etc.

Modern contract generators and contract lifecycle management (CLM) systems exemplify this archetype. A database of pre-approved clauses is available, which are selected through automated rules (auto-templates), drag and drop, or by a user working their way through a guided questionnaire. The database manager can then provide suggested clauses (users who used clause A may also want clause B), prohibit combinations (it may be illegal to combine X with Y), keep the database under centralized version control, roll out updates quickly and perform statistical analysis on usage and gauge legal exposure across an entire organization.

As I see it, there are two ways to integrate LLMs into modular drafting:

Insofar as modular drafting with LLMs is performed in a manner similar to templating (e.g. LLM-generating building blocks and then custom-combine or LLM-combine them), the issues are the same. Each final document would need to be expert-checked, defeating the speed and cost benefits of modular drafting.
However, LLMs can also be used elsewhere in the pipeline, such as through suggesting pre-approved modules and extracting variables from source documents (feature extraction) to produce a first draft. It is my understanding that these steps are integrated into the workflow underlying FRAUKE, an electronic drafting assistant for judgments in air passenger rights cases currently being trialled in the district court (Amtsgericht) of Frankfurt (Germany). However, I have no further insight into the project and do not know if FRAUKE uses an LLM or a different machine learning approach.

The second option does have some merit. I am slightly concerned about the use of LLMs as feature extractors, since standard annotated NLP workflows appear to perform better and errors are more easily corrected with updated annotations (Honnibal 2023). However, LLM results might be acceptable, since some human review is required in any case.

LLMs might also come in handy where there are no rule-based methods for extracting the desired features and the setting is zero-shot or few-short learning. Suggesting pre-approved modules is probably fine in terms of legal quality (although it might miss problematic combinations) and might also be acceptable in a normative sense, since both the modules and the final result are subject to human review.

Bespoke Drafting Link to heading

Bespoke drafting is the most time-consuming type of legal drafting, making the LLM shortcut an especially attractive option. However, it is also the most challenging variant, since the underlying problem is probably rare or unique and requires a custom solution. Examples include international treaties, acts of parliament, superior court judgments, high-value contracts and academic papers, but also less dramatic work products with more interpersonal character such as client correspondence, meeting records or memos.

Since a custom draft requires custom review and editing of the whole document in any case, using an LLM to produce a first draft or provide helpful suggestions might prove beneficial. Having an LLM produce an unchecked final draft is almost always a terrible idea, since its stochastic nature, disregard of truth and lack of legal subroutines will result in a document that is not fit for use. In the judiciary, a human decision (at least in Germany) is always required.

However, drafting is only part of the workload. The more complex types of text (e.g. judgments, normative documents) will need much time spent debating and negotiating their content with other humans, work that cannot easily (or at all) be delegated to an LLM. They may also be very research-intensive, so the actual bottleneck in document production may not even be the drafting step. In many cases drafting and research are also a combined activity and cannot be split, with one part being delegated. Complex correspondence or legal memos are no different. The actual time saved might be less than hoped for, particularly since LLM ouput can also be tricky to get right. Spending an hour on prompt engineering instead of drafting and research provides no benefit at all.

Simple correspondence might be a good use case for an LLM, since it requires little to no research and can be verified quickly. However, there is the very real danger of lawyers ‘fluffing’ up their texts with an LLM, which are then fed into the busy respondent’s LLM to compress them back to a readable size. The result is a loss of information, waste of time, money and energy, since LLMs solve problems they themselves created in the first place.

All of this applies to business, legal practice and public office. The norms governing the use of LLM-assistance in academic work (as opposed to LLMs as an object of study) are still very much in flux, but since there are no reliable ways of detecting generated text, I assume the use of LLMs will be tolerated in the long-term, since there is no way of barring them without unacceptable collateral social damage. I do feel that LLMs are at their weakest in academic settings, since they remix known content in a plausible manner but without being properly embedded in accepted bodies of knowledge or access to empirical reality (no, the internet is NOT an acceptable substitute for empirical reality). Although, they might be a boon to English as a Second Language speakers (ESL), who could gain access to more sophisticated phrasings, albeit likely at the price of misrepresenting their own work.

Mixed Drafting Link to heading

While you do encounter these pure archetypes, legal drafting is just as often a combination of two or three of them. You might combine several modular paragraphs with some bespoke drafting to build a full template for a certain class of tasks and then add some further bespoke drafting to individualize each document to a certain set of circumstances. This happens in mass litigation, but also in contract generation, if you have the authority to add custom modifications (as in consumer or small business settings).

With mixed drafting, one encounters all of the dangers of LLMs at the same time.

Furthermore, one risks the imposition of bias through the helpful suggestion of specific language. LLM training data is what humans have written and published, which is, naturally, biased and tends to privilege men over women, majorites over minorites, white people over BIPOC, able-bodied over the disabled, neuro-typical over neuro-divergent and so on. The sensitive, experienced and skilled writer may spot these subtle influences and correct them, but the less sensitive, the less experienced and the less skilled an LLM-assisted legal drafter is, the more these subtle biases will creep into their writing.

Or maybe they will do so anyway? Legal writing is often unnecessarily obtuse in an attempt by inexperienced writers to signal knowledge through formality (I have been guilty of this many times). This is not a new trend, but I assume LLMs would reinforce it.

LLMs reproduce averages and defaults. We know from decades of working with computers, that very few people are willing and able to challenge the defaults of programs.

Summarization Link to heading

The automatic summarization of texts seems useful at first glance. I have often been assured by business leaders that they would take an imperfect summary any day over having to read lengthy documents. However, as we know from academic papers, the devil is in the detail. Summarization is actually a form of information retrieval, but limited to a single document instead of a full database. The person reading the summary wants to know what is actually in the document (truth), but also what is most important (relevance).

Therefore, much of the conderations regarding information retrieval and research also apply to summarization. Relevance is human-made and can change in the blink of an eye due to outside circumstances, changes that are difficult to incorporate into an LLM quickly. Maybe this will change with smaller model sizes?.

Using an LLM for summarization seems feasible to discover the rough subject of the document, but I would not rely on it to extract key holdings or arguments.

Programming for Lawyers Link to heading

Programming for lawyers is a large maybe. I’ve been assured by friends and colleagues that using ChatGPT or GitHub Copilot is great for learning and assisted coding and has helped them very much. On the flip side, I’ve also seen someone on Mastodon mention (third-hand account) how a student deleted important data by overlooking an “rm -rf *” statement (or similar) in the generated code. For the non-programmers: this command recursively force-deletes all files in the current and all sub-directories where it is executed.

Personally I like puzzling out coding solutions myself and avoid even the basic assistance provided by IDEs like RStudio (I code in Emacs). So I am probably very much the wrong person to sweeping claims about the usefulness of LLM-assisted coding.

But I do have one point: the challenge in writing code is not writing the code. It’s figuring out what you want the code to do. Especially if it’s statistical programming and the code is doing some complicated math.

If you want to generate a plot and format it nicely, that might work. But it might also do something wrong and unless you understand rather well what the code does, the resulting diagram might completely mislead you.

It’s even worse with statistical modeling. Consider a simple case: linear regression. Of course you can have it create the model for you based on a rough description, run the code and explain the results. But should you trust its output? I think not.

To be fair, there is enough linear regression content on the internet that it might do a fair job on toy problems. But as soon as you move off the well-trodden path (as you most likely will need to do for a new problem), all bets are off. There are also many more considerations to modeling, such as feature extraction, feature selection, data quality, measurement error, missing data, causal diagrams, researcher degrees of freedom and even the elephant in the room of whether frequentist statistics is appropriate to your research question at all.

Statistical programming already suffers from too much mindless ritual and too little careful thought. I doubt that LLMs will improve the situation.

Conclusion Link to heading

LLMs and transformer technology are here to stay, although I have serious doubts as to whether they will improve the practice or business of law as much as their proponents claim. I do see some modest opportunities, but nowhere near the potential that evangelists preach. Mostly, I think, people will be disappointed.

If you have well-functioning data infrastructure, a competent data team and well-trained end-users there should be a fair number of possibilities one could explore. But who can afford all three?

The more complex a legal task, the more likely we are to look for some kind of magic to solve our problems. But the quality of Transformers appears to degrade with increasing task complexity (Dziri et al 2023 / pre-print). We also stand at the cusp of an era in which LLM-generated text will vastly outstrip human text, probably causing catastrophic failures in future LLMs due to training new models on auto-generated LLM text (Shumailov et al 2023 / pre-print). Down-sizing models also appears to be more difficult than first thought (Gudibande et al 2023 / pre-print). Securing models is unsuprisingly problematic, given their size, opacity and complexity (El-Mhamdi et al 2023 / pre-print). The hype around LLMs as some sort of novel intelligence also completely ignores the fact that they are more akin to data-washing, hiding the immense amount of “data labor” performed by low-paid gig workers or outright data theft from skilled authors and artists that goes into providing their fuel (Li et al 2023 / pre-print)

If you take into account these limitations in producing quality output, the sheer amount of work and therefore cost (infrastructure, data labor, end-user training) required to make LLMs perform well and their likely degradation in the future, this will probably limit their medium-term to long-term applications.

Naturally, technology as a whole always improves, but this does not mean that this specific technology will. Remember Blockchain, which started as the future of finance and is still as energy-hungry and inefficient as ever. Despite all the advances in AR/VR technology the Metaverse still remains useless and in solid “Ok, Boomer” territory.

Of course everything might turn out differently. Predictions are hard — especially if they concern the future.³

Acknowledgements Link to heading

I am very grateful to the work that Emily Bender, Timnit Gebru, Margaret Mitchell and their colleagues at the Distributed AI Resarch Institute (DAIR) have been doing in this space. I do not know them personally, but their work has significantly informed and influenced my thinking on the subject of this essay.

This essay contains no LLM-generated text and especially no tiresome “did you notice that what you were reading was generated by AI” tropes.

This is is attributed to someone, but I can’t find a primordial citation. ↩︎
To be fair, I wrote this text in Emacs with the Wombat theme, so it is more of a very dark gray screen. ↩︎
Attributing this aphorism opens up an interesting rabbithole: https://quoteinvestigator.com/2013/10/20/no-predict/ ↩︎

[New Recording/Slides] International Legal Data in Action (FGV Rio Workshop)

Thu, 23 Mar 2023 00:00:00 +0000

Info

[Open Access] Slides for ‘International Legal Data in Action’ (Zenodo 2023)
[Open Access] Recording of Panel I (FGV Workshop 2023)
[Open Access] Recording of Panel II (FGV Workshop 2023)

FGV Workshop Link to heading

Last week I delivered an online presentation titled ‘International Legal Data in Action: Ideas and Applications for the ICJ and PCIJ’ during Panel II of the Fundação Getulio Vargas (FGV) Workshop “Transforming the Role of International Courts and Tribunals in a New Era of Adjudication” on 16 March 2023. The event was streamed live on YouTube and links to recordings are included in the infobox above. I have also published the slides for the presentation, also linked in the infobox.

Abstract Link to heading

There can be no Data Science without data. However, the availability of legal data sets, particularly corpora in international law, has been rather limited until recently. In this presentation I discuss two new and high-quality international legal sets for the International Court of Justice (ICJ) and the Permanent Court of International Justice (PCIJ). These corpora collect the full-texts and metadata for all majority and minority opinions of both Courts from 1922 to 2022 and are fully compatible with each other. The ICJ corpus is updated twice a year with new data.

I further outline research questions and methods to put this international legal data into action. These include doctrinal analysis, citation analysis, social network analysis and geospatial analysis of full-texts and metadata. References to relevant papers provide an introduction to the research problems and methdologies for researchers who wish to tackle them. The presentation closes with a call for replication though the publication of source code and data.

Download Data Sets Link to heading

Academic Paper Link to heading

Fobbe, S. (2022). Introducing Twin Corpora of Decisions for the International Court of Justice (ICJ) and the Permanent Court of International Justice (PCIJ). Journal of Empirical Legal Studies, 19(2), 491-524. https://doi.org/10.1111/jels.12313

[New Article] The Length of Proceedings before the German Constitutional Court

Mon, 02 Jan 2023 00:00:00 +0000

Info

[Open Access] Fobbe, Seán. “Sind zwanzig Jahre zuviel? Die Dauer von Verfahren vor dem Bundesverfassungsgericht” (RECHTS|EMPIRIE 2022)
[Open Access] Source Code (Zenodo)

Background Link to heading

In December I published an analysis of the length of proceedings before the German Federal Constitutional Court (BVerfG) in RECHTS|EMPIRIE, a German science blog. The links in the info box above will direct you to the original text of the article and the source code (in R). Below is an English summary.

English Summary Link to heading

Introduction Link to heading

The German Federal Constitutional Court (BVerfG) is not only a guardian of justice, but also a guardian of time. Judgments in court proceedings should be just, but they should also be prompt. This maxim has been pithily summarized as “justice delayed is justice denied” (Gladstone) or “swift justice is the sweetest” (Bacon). In German it is known as the “Beschleunigungsgrundsatz” and enshrined in domestic statutes, the constitution and the European Convention of Human Rights (ECHR).

The Role of the Beschwerdekammer Link to heading

I briefly discuss the role and jurisprudence of the Beschwerdekammer, the special panel of the BVerfG that reviews the length of constitutional proceedings. Since its creation in 2011 it has ruled on few cases and published only seven decisions. Based on the case law of this special panel it would appear that, regarding its own constitutional proceedings, the BVerfG generally considers a length of five to seven years to be acceptable.

The role and composition of the Beschwerdekammer represents a prima facie violation of the principle nemo judex in sua causa (no one can be a judge in their own cause), as the judges rule on the quality of their own professional conduct and that of colleagues in a very small court. There exist only limited safeguards, such as the exclusion of the rapporteur of an impugned decision from Beschwerdekammer proceedings.

However, there do not appear to be any alternative options that would not undercut the special role of the BVerfG in the German constitutional framework. The endless saga surrounding the new disciplinary panel of the Polish Constitutional Court demonstrates the danger of external interference with an apex court.

A Silver Lining Link to heading

There is a silver lining to this situation: if BVerfG judges are judges in their own professional causes, then an empirical analysis of its activities will allow us to augment our understanding not only of the practice of the court, but also of its normative stance on the acceptable length of constitutional proceedings.

The official statistics of the BVerfG are quite terse with regard to the actual length of proceedings. Specifics are only available for constitutional complaints and do not cover durations of more than three years.

To gain a better understanding of the court’s practice I turn to the Corpus der Entscheidungen des Bundesverfassungsgerichts (CE-BVerfG) (Fobbe 2022), which documents all published BVerfG decisions since 1998, plus metadata. This metadata includes the year in which proceedings were initiated and the date the judgment was rendered. From these I calculate an estimate of the length of proceedings in years. Several notable limitations are discussed and sensitivity analyses performed (see source code for full details).

Results Link to heading

The data consists of 8,407 published decisisons. After excluding procedural decisions (e.g. costs), 7,700 decisions remain. I visualize the frequency distribution of length in years via bar plots for all decisions, senate decisions (important cases) and panel decisions (all other cases). It is immediately obvious that many decisions take more than three years.

44 decisions even hint at proceedings of 10 or more years. A single outlier decision was handed down after slightly less than 21 years.

The median for senate proceedings is 2 years and 1 year for panels, with a worst case measurement error of 4 months. The mean for senate proceedings is 2.87 years and 1.35 years for panel proceedings, with negligible measurement error.

After a close reading of the full text, the single 21-year-decision appears to be a curiosity of legal history, not a scandal. The petitioner challenged an abuse fee (Missbrauchsgebühr) levied by the BVerfG. However, such decisions are final and there exist no formal remedies to challenge them. The application was clearly without merit. This incredible delay in an absurd situation is reminiscent of a famous play linked to the Theatre of the Absurd: Waiting for Godot.

Bar Plot Link to heading

The x-axis shows length of proceedings in years, the y-axis the number of decisions. This diagram includes all decisions from both senates (8 judges) and panels (3 judges).

Experimental Visuals Link to heading

The article closes with experimental visualizations: Chord diagrams. These have known limitations (the human eye being unable to judge angles accurately, difficult to interpret without assistance), but can represent the data in the original in-year/out-year format and are able to show the change over time in length of proceedings. The diagrams are interpreted as an example for future users.

Length of Proceedings (Senates) Link to heading

Length of Proceedings (Panels) Link to heading

Open Data Impact Award for the ICJ Corpus

Sat, 12 Nov 2022 00:00:00 +0000

Info

Open Data Impact Award for the ICJ Corpus Link to heading

The 2022 Open Data Impact Award was a surprise, but a pleasant one! Thank you, Stifterverband, for awarding third place and 5,000 € to the Corpus of Decisions: International Court of Justice (CD-ICJ), as well as inviting me to present my project ‘Foundations of Legal Data Science’ in Berlin! I’ve uploaded the slides and notes to Zenodo. Includes a peek at an exciting new project!

The award-winning Corpus of Decisions: International Court of Justice (CD-ICJ) collects all published decisions and appended opinions of the International Court of Justice (ICJ), the judicial organ of the United Nations and one of the most consequential courts in international law.

Want to jump in right away? Download it here! Want to work with more legal data? Choose from 16 different data sets listed on my website.

📷 Photo by David Ausserhofer

OpenSenseMap and Risk-Based Authentication Link to heading

Congratulations to my fellow awardees, who made this evening (and the after-party) a very special occasion!

🥇 First place and 15,000 € went to the OpenSenseMap project, headed by Professor Angela Schwering, Dr. Thomas Bartoschek and Mario Pesch. OpenSenseMap is an incredible global database of environmental sensor data, containing more than 11 billion measurements from over 10,000 sensors. Give OpenSenseMap a spin with its beginner-friendly web interface!

🥈 Second place and 10,000 € went to Stephan Wiefling and Professor Luigi Lo Iacono for creating the Login Data Set for Risk-Based Authentication, a synthetic compilation of more than 33 million realistic authentication attempts from over 3.3 million users. It may re-define login Risk-Based Authentication security measures for millions of websites on the internet.

📷 Photo von David Ausserhofer

Left to right: Andrea Frank (Stifterverband), Seán Fobbe, Mario Pesch, Professor Dr. Angela Schwering, Dr. Thomas Bartoschek, Anna Held (Stifterverband), Stephan Wiefling and Professor Dr. Luigi Lo Iacono.

[Podcast] Talking Legal Tech

Wed, 09 Nov 2022 00:00:00 +0000

Info

Talking Legal Tech Podcast — Legal Data Science with Seán Fobbe (November 2022)

Felipe Molina Gaviria and I joined forces on the Talking Legal Tech Podcast and spent a fun hour discussing Legal Data Science, peace research and digital detective work. The podcast is available on a wide range of platforms via the link up top in the info box:

Anchor.fm
Apple Podcasts
Castbox
Google Podcasts
Overcast
Pocket Casts
Radio Public
Spotify

Please note, the language of the podcast is German. I haven’t recorded an episode on Legal Data Science in English yet, but if you happen to host a podcast, let me know and perhaps we can make it happen!

Have more questions after listening to the episode? Write me!

Moving from Twitter to Mastodon

Tue, 08 Nov 2022 00:00:00 +0000

Info

New Mastodon Profile — @seanfobbe@fediscience.org

Moving to Mastodon Link to heading

Now that Twitter is burning down faster than anyone expected, I joined the mass migration of academics to Mastodon and opened an account on the fediscience.org instance last Friday.

So far I have been enjoying Mastodon immensely. As a matter of fact, I’ve had more high-quality exchanges with colleagues and people outside of my bubble in a weekend than I did during the two years I spent on Twitter. There’s something rather special about the Mastodon community, even with the massive influx of new users, and I hope we can preserve that community spirit.

I’ll stay a while longer on Twitter to redirect users to my new Mastodon profile, but I assume that I’ll be engaging less and less as time goes on. The handling of this takeover is one of the worst messes I’ve seen in a while and I would not be surprised if Twitter were to end up like MySpace soon.

Advice for New Mastodon Users Link to heading

There are many fantastic resources for new users out there and more are being written daily. These are some of my favorites:

[Update] ICJ and PCIJ Corpora

Sun, 11 Sep 2022 00:00:00 +0000

Info

[Open Access] Corpus of Decisions: International Court of Justice (CD-ICJ)
[Open Access] Corpus of Decisions: Permanent Court of International Justice (CD-PCIJ)
[Open Access] Introducing Twin Corpora of Decisions for the International Court of Justice (ICJ) and the Permanent Court of International Justice (PCIJ) (JELS 2022)

Notable Changes Link to heading

September saw scheduled updates published for the Corpus of Decisions: International Court of Justice (CD-ICJ) and the Corpus of Decisions: Permanent Court of International Justice (CD-PCIJ). In addition to a number of minor bug fixes and enhancements both corpora were fully recompiled with Tesseract 5, yielding an even better OCR rendering of old decisions.

Compared to the 50.19% reduction in unique tokens for the English version of the CD-ICJ, as reported in the Journal of Empirical Legal Studies, the reduction now stands at 51.15% compared to the original OCR published by the Court. Faulty OCR can cause a large number of unique or rare tokens that may bias or otherwise impede statistical analysis, particularly bag-of-words models. Please see the JELS article for more on OCR quality control. Improvements in the French version of the CD-ICJ and both language versions of the CD-PCIJ are comparable.

Also, the CD-ICJ now includes cases up to General List No 183 (Germany v Italy) and documents published before September 2022.

For full details please see the changelog on the relevant Zenodo pages and in each Codebook.

[Update] German Legal Corpora

Sat, 10 Sep 2022 00:00:00 +0000

Info

[Open Access] German Legal Corpora on Zenodo (Fobbe)

Overview Link to heading

The following open access data sets received (significant) updates between June and September 2022:

This includes full recompilation of data sets with the newest decisions. For full details please see the changelog on the relevant Zenodo pages and in each Codebook.

Please submit bugs and feature requests to the relevant Issue Tracker on GitHub.

[New Paper] Introducing Twin Corpora of Decisions for the International Court of Justice (ICJ) and the Permanent Court of International Justice (PCIJ)

Fri, 29 Apr 2022 00:00:00 +0000

Info

[Open Access] Introducing Twin Corpora of Decisions for the International Court of Justice (ICJ) and the Permanent Court of International Justice (PCIJ) (JELS 2022)
[Open Access] Corpus of Decisions: International Court of Justice (CD-ICJ)
[Open Access] Corpus of Decisions: Permanent Court of International Justice (CD-PCIJ)

In this article I present the first two of a new series of open and high-quality international legal data sets: comprehensive, fully reproducible, human- and machine-readable open access collections covering one hundred years of case law of the primary judicial organs of the United Nations and the League of Nations: the Corpus of Decisions: International Court of Justice (CD-ICJ) and the Corpus of Decisions: Permanent Court of International Justice (CD-PCIJ).

Each corpus is designed to capture in its entirety the published case law of its eponymous Court, including majority opinions (judgments, advisory opinions and orders), but also the minority opinions annexed to each decision (declarations, separate opinions and dissenting opinions). The corpora are enriched with useful metadata to enhance text-as-data research and enable stand-alone metadata analyses. While each corpus can stand on its own, the twin corpora are designed to be perfectly interoperable for the purposes of analyses that wish to treat the ICJ and PCIJ as a continuous entity.

The most recent versions of the corpora will always be available open access at doi.org/10.5281/zenodo.3826444 (CD-ICJ) and doi.org/10.5281/zenodo.3840479 (CD-PCIJ).

Corpus of Decisions: International Court of Justice (CD-ICJ) Link to heading

The Corpus of Decisions: International Court of Justice (CD-ICJ) collects and presents for the first time in human- and machine-readable form all published decisions of the International Court of Justice (ICJ). Among these are judgments, advisory opinions and orders, as well as their respective appended minority opinions (declarations, separate opinions and dissenting opinions). The International Court of Justice has kindly made available these documents on its website.

The International Court of Justice (ICJ) is the primary judicial organ of the United Nations and one of the most consequential courts in international law. Called the ‘World Court’ by many, it is the only international court with general thematic jurisdiction. While critics occasionally note the lack of compulsory jurisdiction and sharply limited access to the Court, its opinions continue to have an outsize influence on the modern interpretation, codification and wider development of international law.

Every international legal textbook covers the workings and decisions of the Court in extenso and participation in international moot courts such as the Philip C. Jessup Moot Court without regular reference to and citation of the International Court of Justice’s decisions is unthinkable.

Features

Fully compatible with the Corpus of Decisions: Permanent Court of International Justice (CD-PCIJ)
27 variables
Public Domain (CC Zero 1.0)
Open and platform independent file formats (PDF, TXT, CSV)
Extensive Codebook
Compilation Report explains construction and validation of the data set in detail
Large number of diagrams for all purposes (see the ‘ANALYSIS’ archive)
Diagrams are available as PDF (for printing) and PNG (for web display)
Tables are available as CSV for easy readability by humans and machines
Secure cryptographic signatures (SHA3-512 and RSA-4096)
Publication of full source code (Open Source)

Corpus of Decisions: Permanent Court of International Justice (CD-PCIJ) Link to heading

The Corpus of Decisions: Permanent Court of International Justice (CD-PCIJ) collects and presents for the first time in human- and machine-readable formats all documents of PCIJ Series A, B and A/B of the Permanent Court of International Justice (PCIJ). Among these are judgments, advisory opinions, orders, appended minority opinions, annexes, applications instituting proceedings and requests for an advisory opinion. The International Court of Justice, the successor of the PCIJ, has kindly made available these documents on its website.

The Permanent Court of International Justice (PCIJ) was the primary judicial organ of the League of Nations, the ill-fated predecessor of the United Nations, which existed from 1920 to 1946. Nonetheless, as the first international court with general thematic jurisdiction, the PCIJ influenced international law in profound ways that are still felt today. Every lawyer who sets out on the path of international law encounters epoch-defining opinions such as the Lotus and Factory at Chorzów decisions, but the Court’s lesser-known jurisprudence and the appended minority opinions offer many more ideas and legal principles which are seldom appreciated today.

Features

Fully compatible with the Corpus of Decisions: International Court of Justice (CD-ICJ)
29 variables
Public Domain (CC Zero 1.0)
Open and platform independent file formats (PDF, TXT, CSV)
Extensive Codebook
Compilation Report explains construction and validation of the data set in detail
Large number of diagrams for all purposes (see the ‘ANALYSIS’ archive)
Diagrams are available as PDF (for printing) and PNG (for web display)
Tables are available as CSV for easy readability by humans and machines
Secure cryptographic signatures (SHA3-512 and RSA-4096)
Publication of full source code (Open Source)

2022 Russian Invasion - How to Help Ukraine

Fri, 25 Feb 2022 00:00:00 +0000

Info

This article will be updated as the situation evolves. Please check back frequently.

Overview Link to heading

On 24 February 2022 armed forces of the Russian Federation invaded the sovereign territory of Ukraine in clear contravention of international law, particularly the prohibition of the use of force set down in Article 2 (4) UN Charter, the cornerstone of the modern international order. There is no justification for this attack on the political independence and territorial integrity of Ukraine.

I stand in solidarity with the Ukrainian people and condemn this illegal use of force and war of aggression in the strongest possible terms. To help alleviate the suffering caused by the conflict I’ve compiled a list of information that anyone who wishes to aid the people of Ukraine through donations or volunteering their time and skills may find useful.

This article contains information on:

Trustworthy News Sources
Peace Protests
Humanitarian Assistance
Supporting the Ukrainian Armed Forces

#StandWithUkraine

News Sources Link to heading

It is said that truth is the first victim of war, but it is also the first victim of social media. Here I have selected a few news sources you might find more reliable than Twitter and Facebook:

Kyiv Independent — Independent Ukrainian News Website in English
Dekoder — Russian Journalism translated into German
Institute for the Study of War: Ukraine Conflict Updates — Daily summaries of the strategic and tactical military situation in Ukraine

Peace Protests Link to heading

Taking to the streets and calling for peace is one of the time-honored democratic measures to end wars.

Stand with Ukraine — Regularly updated list of peace protests on behalf of Ukraine
Circle for Peace — Stand up for peace by drawing a peace symbol on the ground and protesting in it for as long as you can

Humanitarian Assistance Link to heading

Regional spillover from such a major conflict is inevitable so I have included organizations working both in Ukraine and in the wider region. Please note that these organizations will likely need your support for years to come in order to deal with the humanitarian fallout — not just during the immediate crisis.

Ukraine Link to heading

Libereco – Partnership for Human Rights e.V. and their Ukraine Emergency Aid Fundraiser (cooperation with Vostok SOS) — Humanitarian Aid and Human Rights
Vostok SOS and their Ukraine Emergency Aid Fundraiser (cooperation with Libereco) — Humanitarian Aid and Human Rights
Support Hospitals in Ukraine — Medical Aid
United Help Ukraine — Medical/Humanitarian Aid
Caritas Ukraine — Humanitarian Aid
Ukraine-Hilfe Berlin — Medical/Humanitarian Aid
Voices of Children — Children
Care in Action and their Ukraine Emergency Fundraiser — Children and Orphans
Mission Lifeline — Small convoy traveling to Ukrainian border to help refugees

Poland Link to heading

Polish Humanitarian Action PAH and the PAH Ukraine Fundraiser
Ocalenie Foundation — Volunteers can sign up to help refugees in Poland
Polish Center for International Aid — Fundraiser for Humanitarian aid, evacuation aid, psychological aid for children
Animal Protection Inspectorate — Fundraiser to save Animals — Contact: sekretariat@dolnoslaski-ioz.pl
Dialog Foundations — Refugee Protection and Fundraiser to create a house for Ukrainian families in Poland
Polish Red Cross
Polish Medical Mission and their Ukraine Fundraiser on Facebook
Foundation Pomagam.pl in coordination with ‘Nasz Wybór’ (‘Our Choice’) Foundation
Open Dialogue Foundation and their Ukraine Emergency Fundraiser

Germany Link to heading

Elinor Network/GLS Bank/Ecosia: “Gastfreundschaft Ukraine” — via this form you can offer short-term accommodation in Germany to Ukrainian refugees

Switzerland Link to heading

Ukrainian Embassy in Bern/Switzerland [Website in German] — Call from Ukrainian embasssy in Bern to donate medical supplies (Mo–Fr, 4–7 pm), please contact swisshelpukraine@gmail.com in advance
I want to Help Ukraine (Geneva/Switzerland) — Donate medical and other humanitarian supplies
Permanent Mission of Ukraine in Geneva (Facebook) — Donate medical and other humanitarian supplies

Europe Link to heading

Aid4Ukr — Guide for Ukrainian Refugees — Immigration Information for Ukrainian Refugees
Immigration 4 Ukraine — Lawyers can register to offer pro bono advice and Ukrainians can register to receive legal advice on immigration
Pan-european decentralized offers of support for Ukrainian refugees

International Organizations (IGO and NGO) Link to heading

Global Link to heading

International Support for Students and Researchers from Ukraine
Labs supporting Ukrainian Scientists
Ukrainian Tech Collective — Facilitates skill swaps so Ukrainians who are currently protecting their country still get paid and can support their families.

Ukraine Armed Forces Special Account Link to heading

Note

Following the declaration of martial law the National Bank of Ukraine opened a special fundraising account to support the Armed Forces of Ukraine. The account accepts donations in UAH, EUR, USD, GBP, CHF, AUD, CNY, CAD and JPY.

Special Fundraising Account for Ukrainian Armed Forces (Official Information)

I take this unprecedented step in promoting this account as a peace-loving citizen and lawyer because the 2022 Russian invasion of Ukraine represents an illegal war of aggression and a flagrant violation of international law, particularly the prohibition of the use of force (Article 2 (4) UN Charter). The right of Ukraine to act in self-defence under Article 51 of the UN Charter is legally and morally without reproach.

Quality of Sources Link to heading

Note

The information contained in this article was received from different sources in my network. I have tried to independently verify the credibility of each organization whereever possible but this is a dynamic and fast-moving situation, so errors are possible.

If you have justified concerns about any of these organizations, please get in touch

[New Data Sets] Full ICJ and PCIJ Jurisprudence (1921-2021) as Comprehensive Data Sets

Thu, 24 Feb 2022 00:00:00 +0000

Info

[Open Access] Corpus of Decisions: International Court of Justice (CD-ICJ)
[Open Access] Corpus of Decisions: Permanent Court of International Justice (CD-PCIJ)

Overview Link to heading

Last week I released full open access collections of all International Court of Justice (ICJ) and Permanent Court of International Justice (PCIJ) judgments, advisory opinions and other decisions:

[Open Access] Corpus of Decisions: International Court of Justice (CD-ICJ)
[Open Access] Corpus of Decisions: Permanent Court of International Justice (CD-PCIJ)

You can now download all opinions for each Court with two clicks. The PDF versions have been enhanced with neural networks to vastly improve the text layer for copy & paste and searching within documents (Ctrl+F). All files have received improved filenames (including shortened case names, applicants/respondents and the stage of proceedings). I have also included an additional variant consisting only of the majority opinions to assist busy practitioners.

There’s much more to say since the data sets are intended for advanced statistical analysis, but you’ll find details on that in the documentation and the forthcoming Journal of Empirical Legal Studies (JELS) paper. The data sets were created with the permission of the Registrar of the International Court of Justice.

Feel free to share with any colleagues interested in international law!

Corpus of Decisions: International Court of Justice (CD-ICJ) Link to heading

Features

Fully compatible with the Corpus of Decisions: Permanent Court of International Justice (CD-PCIJ)
27 variables
Public Domain (CC Zero 1.0)
Open and platform independent file formats (PDF, TXT, CSV)
Extensive Codebook
Compilation Report explains construction and validation of the data set in detail
Large number of diagrams for all purposes (see the ‘ANALYSIS’ archive)
Diagrams are available as PDF (for printing) and PNG (for web display)
Tables are available as CSV for easy readability by humans and machines
Secure cryptographic signatures (SHA3-512 and RSA-4096)
Publication of full source code (Open Source)

Corpus of Decisions: Permanent Court of International Justice (CD-PCIJ) Link to heading

Features

Fully compatible with the Corpus of Decisions: International Court of Justice (CD-ICJ)
29 variables
Public Domain (CC Zero 1.0)
Open and platform independent file formats (PDF, TXT, CSV)
Extensive Codebook
Compilation Report explains construction and validation of the data set in detail
Large number of diagrams for all purposes (see the ‘ANALYSIS’ archive)
Diagrams are available as PDF (for printing) and PNG (for web display)
Tables are available as CSV for easy readability by humans and machines
Secure cryptographic signatures (SHA3-512 and RSA-4096)
Publication of full source code (Open Source)

Open Sourcing my Linux Configuration

Wed, 23 Feb 2022 00:00:00 +0000

Info

[Open Access] My personal Linux configuration on GitHub

Ever wondered on what kind of setup I run my computational analyses? I’ve already written about some of my preferred tools, particularly Fedora Linux and Debian. I decided to take the next step and open source the full range of installer scripts and dotfiles I use to customize my development machines. These include:

A setup and config script
Installer scripts for Fedora and Debian packages (includes dependencides for common R packages)
Installer script for my preferred R packages
A script to clone all my public git repositories

Dotfiles are small text files that store configuration settings on Linux systems and are easily portable. They are aptly so named because their names starts with a dot (.) to let the system know that these are hidden files. You can store them in a GitHub repo like I do and when you setup a new system they are easily copied to make yourself instantly at home.

I use a neat trick I picked up somewhere on the internet to replace all existing config files with symbolic links to the git repository, so I don’t need to constantly re-copy them from the repo when I update my config.

Please note that in almost all cases you should not simply run these scripts as-published on your machine, but instead use them as an inspiration to create your own. Especially the first script will execute irreversible actions by deleting all current dotfiles of interest and replacing them with symbolic links to the ones in my git repo.

[New Paper] Final Report of Investigation into Cultural Heritage Destruction during the Islamic State's Genocide against the Yazidis

Mon, 15 Nov 2021 00:00:00 +0000

Info

[Closed Access] Read the final peer-reviewed paper: doi.org/10.1163/9789004466180_006
[Open Access] Read the full manuscript: zenodo.org/record/5595882
[Open Access] Read the original report: doi.org/10.5281/zenodo.3826125

JOINT PRESS RELEASE Link to heading

Seven years after the beginning of the 2014 genocide committed by the Islamic State against the Êzidîs (‘Yazidis’ or ‘Yezidis’) the final report of a joint investigation conducted by RASHID International, Yazda and the Endangered Archaeology in the Middle East and North Africa Project (EAMENA) has been released. The results were published in Volume 5 of the peer-reviewed Asian Yearbook of Human Rights and Humanitarian Law (AYHR), an international human rights periodical edited by Professor Javaid Rehman, the UN Special Rapporteur on the human rights situation in Iran.

International attention concerning the genocide has generally focused on murder, slavery and sexual exploitation. We analyzed the destruction of Êzidî tangible and intangible cultural heritage as a significant facet of the Islamic State’s policy of ethnic cleansing and genocide. Evidence of destruction is collected and presented in context with other criminal acts.

The Êzidî are an endogamous community at home in northern Iraq for whom faith and ethnic belonging are inextricably linked. Belief in God and Tawûsê Malek (the highest angel), and reverence for Lalish as the holiest place on earth are the defining features of the Êzidî faith. Historic and sacred places are an essential part of the Êzidî identity and are considered vital to life by the local population. The Islamic State made no secret of its intention to eradicate the Êzidî community and commenced a policy of ethnic cleansing and genocide on 3 August 2014. All victims were abused and tortured. Male Êzidîs above the age of 12 were killed. Female Êzidîs were enslaved and traded in a complex and public network of sexual slavery. Boys were trained in ISIS camps and militarized. Those who fled to Mount Sinjar were besieged in order to ensure death from starvation, thirst and the blazing sun. Bases of economic support, such as olive groves and irrigation wells, were systematically destroyed and many areas of the Êzidî homeland were sown with landmines and improvised explosive devices (IEDs) to prevent the population from returning.

The peer-reviewed AYHR publication provides original research, evidence and context on the destruction of Êzidî tangible cultural heritage in the Bahzani/Bashiqa and Sinjar areas of northern Iraq. We present satellite imagery analysis conducted by the EAMENA project, drawing on data provided by Êzidî representatives. According to the Department of Yazidi Affairs in the Ministry of Awqaf and Religious Affairs in the Kurdistan Regional Government 68 Êzidî sites were destroyed by the Islamic State. We consider 16 sites in the Bahzani/Bashiqa area and 8 in the Sinjar area to which access was possible and which could be documented.

The destruction of cultural heritage sites is most often considered a war crime. Several convictions by the International Criminal Tribunal for the former Yugoslavia (ICTY) and the conviction of Malian Islamist Al-Mahdi by the International Criminal Court (ICC) are well-known. However, heritage destruction may also be prosecuted as the crime of persecution, a crime against humanity. Numerous indictments and convictions before international courts attest to the viability of this approach. Finally, grounded in explicit case law of the International Court of Justice (ICJ), destruction of tangible heritage also serves as evidence of the special intent to destroy a protected group under the crime of genocide.

We conclude that the destruction of the cultural heritage of the Êzidî people constituted a war crime, a crime against humanity (persecution) and compelling evidence of genocidal intent. We recommend the consideration of cultural heritage destruction in any prosecution of atrocity crimes, especially the crime of genocide.

Online Availability Link to heading

The final report is intended to be read in conjunction with the original 2019 report, which includes extensive ground imagery and site descriptions to provide additional context.

2021 Final Article (peer-reviewed, subscription required): https://doi.org/10.1163/9789004466180_006
2021 Manuscript (peer-reviewed, open access): https://zenodo.org/record/5595882
2019 Original Report (open access): https://doi.org/10.5281/zenodo.3826125

Contact Link to heading

For further inquiries, please contact Chief Legal Officer Seán Fobbe (legal@rashid-international.org).

Example Citation Link to heading

Fobbe, Seán, Natia Navrouzov, Kristen Hopper, Ahmed Khudida Burjus, Graham Philip, Maher G Nawaf, Daniel Lawrence, Helen Walasek, Sara Birjandian, Majid Hassan Ali, Salim Rashidani, Hassan Salih, Dawood Sulaiman Qari, and Faris Mishko. ‘Cultural Heritage Destruction during the Islamic State’s Genocide against the Yazidis’. In The Asian Yearbook of Human Rights and Humanitarian Law (pp. 111-144). Brill Nijhoff. DOI: https://doi.org/10.1163/9789004466180_006.

About RASHID International Link to heading

RASHID International is a worldwide network of archaeologists and cultural heritage experts dedicated to safeguarding and promoting the cultural heritage of Iraq, ancient Mesopotamia. To assist our Iraqi colleagues, we collect and share information, research and expert knowledge, work to raise public awareness, and both develop and execute strategies to protect heritage sites and other cultural property through international cooperation, advocacy and technical assistance.

We are registered as a non-profit organisation in Germany and enjoys charitable tax-exempt status under German law. We are an organisation in special consultative status with the United Nations Economic and Social Council since 2019. Learn more about our work at www.rashid-international.org.

About Yazda Link to heading

Yazda is a global community-led institution that protects and champions all religious and ethnic minority communities, including Assyrians, Chaldeans, Kakais, Shabak, Turkmen, and Yazidis in Iraq, Kurdistan Region, and Syria. Founded in 2014 at the onset of the genocide perpetuated by Da’esh, also known as the Islamic State in Iraq and Levant (ISIL), against the Yazidis, as well as war crimes and crimes against humanity against other minorities, Yazda leads advocacy and strategic projects in Dohuk, Sinjar, and the Nineveh Plains, as well as global diaspora hubs.

We work hand-in-hand with our partners, including United Nations agencies, key donor governments, the Iraqi and Kurdistan Regional Governments, and instrumental community organizations to ensure justice, accountability, and a sustainable future for all is swiftly reached. We are fervently determined to ensure Never Again, so that no community and people ever face genocide. Learn more about our work at www.yazda.org.

About EAMENA Link to heading

The EAMENA Project rapidly records and evaluates the status of the archaeological landscape of the MENA region in order to create an accessible body of data which can be used by national and international heritage professionals to target those sites most in danger and better plan and implement the preservation and protection of this heritage. The EAMENA project is a collaboration between the Universities of Oxford, Leicester and Durham in the United Kingdom and is supported by the Arcadia Fund and the British Council’s Cultural Protection Fund. Learn more about our work at www.eamena.org.

[New Article] Legal Network Analytics 101

Tue, 05 Oct 2021 00:00:00 +0000

Info

[Open Access] Read the full article (in German): doi.org/10.25527/re.2021.11

[Open Access] Source code and high-res diagrams: doi.org/10.5281/zenodo.5468667

Abstract Link to heading

In this article I discuss a network analytic approach to the internal structure of German laws and regulations, as well as the need for lowering entry barriers to the field. To assist beginners with their first forays into legal network analysis, I provide structural information for almost all German laws and regulations as part of the ‘Corpus des Deutschen Bundesrechts (C-DBR)’. This structural information is available in the widely used GraphML format and includes pre-computed visual representations for initial orientation.

I discuss the advantages and disadvantages of five different kinds of hierarchical graph visualizations (classical dendrogram, two types of circular dendrogram, circle packing and sunburst diagrams) and provide step-by-step instructions to import structural legal data from the C-DBR into graphical analysis software such as GEPHI. I conclude with a detailed breakdown of my methodology and links to the original open source code in the R programming language.

Structure of the German Civil Code Link to heading

[New Article] The International Criminal Court's Draft Policy on Cultural Heritage

Tue, 27 Apr 2021 00:00:00 +0000

Info

[Open Access] Read the full article: doi.org/10.5281/zenodo.4722704

Executive Summary Link to heading

On 23 March 2021 the Office of the Prosecutor (OTP) of the International Criminal Court (ICC) published a Draft Policy on Cultural Heritage dated 22 March 2021. Following its publication the OTP opened a round of external consultations and requested that comments on the policy be submitted by 16 April 2021.

We find the Draft Policy on Cultural Heritage to be, in general, an excellent document, both well-crafted in a technical legal sense and detailing a strong and reasonable strategy to prosecute crimes against cultural heritage.

In Section 2 we highlight several positive aspects of the Draft Policy on Cultural Heritage. The OTP’s cooperation with civil society, as expressed through an expert consultation in 2017, the ability to comment on the Draft Policy in the present external consultation and the strategic mention of civil society are most welcome. The explicit link between cultural heritage and human rights will be a boon to the OTP and analysts of international criminal law.

We commend that the OTP acknowledges that groups and the international community can be victims of crimes against cultural heritage. One of the Draft Policy’s greatest strengths is its holistic definition of cultural heritage, which transcends the traditional focus on physical property and is better able to capture the complexity of culture. We note with appreciation the OTP’s intent to consider the ‘broadest scope of criminality’ and its willingness to conduct a cultural analysis of all of the Rome Statute’s provisions. We support the OTP’s interpretation that war crimes do not include a ‘great importance’ criterion. We strongly encourage the OTP’s consideration of crimes against cultural heritage as possible evidence of the special intent to commit genocide.

In Section 3 we discuss several issues that we feel require further attention by the OTP. We greatly appreciate the OTP’s efforts to add linked citations to open access materials and recommend that it consider citing scholarship with Digital Object Identifiers (DOI). The ‘fundamental rights’ basis of the crime of persecution should be understood to include all human rights, if grave violations have occurred, in line with the jurisprudence of the ICTY.

We provide some additional context on heritage crimes and recommend that the OTP also investigate the illicit trade in antiquities, either as an accessory form of pillage or as a crime against humanity in its own right, with the Islamic State as an example. To assist in this effort we discuss the application of forensic traceable liquids. We further recommend consideration of jurisdiction based on the active personality principle to prosecute transnational actors. The listing system of the 1954 Hague Convention and its Protocols may prove useful in analyzing aggravating factors and command responsibility. The indictment practice of the ICTY is collected and presented for use by the OTP.

Based on Sections 2 and 3 we provide a number of recommendations in Section 4 and conclude with ideas for further reading in Section 5.

About RASHID International Link to heading

RASHID International is registered as a non-profit organisation in Germany and enjoys charitable tax-exempt status under German law. We are an organisation in special consultative status with the United Nations Economic and Social Council since 2019. Learn more about our work at www.rashid-international.org

Info

All RASHID International research is available open access: zenodo.org/communities/rashid-international/

[New Report] Cultural Heritage Destruction during the Islamic State’s Genocide against the Yazidis

Fri, 02 Aug 2019 00:00:00 +0000

Info

[Open Access] Read the full report: doi.org/10.5281/zenodo.3826125
[Open Access] Read the full manuscript of the final paper: zenodo.org/record/5595882
[Closed Access] Read the final peer-reviewed paper: doi.org/10.1163/9789004466180_006

Abstract Link to heading

On 2 August 2019, the eve of the 5th anniversary of the attacks on Sinjar by the Islamic State (IS), RASHID International, Yazda and the Endangered Archaeology in the Middle East and North Africa Project (EAMENA) released the results of their investigation into cultural heritage destruction during the genocide against the Yazidis, in a report entitled ‘Destroying the Soul of the Yazidis: Cultural Heritage Destruction during the Islamic State’s Genocide against the Yazidis’.

Out of the total of 68 sites reported destroyed we consider 16 sites in the Bahzani/Bashiqa area and 8 in the Sinjar area to which access was possible and which could be documented.

Discussions of the genocide committed against the Yazidi people by IS from 2014 onwards have generally focused on murder, slavery and sexual exploitation. In this report we analyze the destruction of Yazidi tangible and intangible cultural heritage as a significant facet of the Islamic State’s policy of ethnic cleansing and genocide. Evidence of destruction is collected and presented in context with other criminal acts.

A revised and updated version of the report was published as a peer-reviewed paper in the Asian Yearbook of Human Rights and Humanitarian Law (AYHL). The AYHL is edited by Professor Javaid Rehman, the United Nations Special Rapporteur on the human rights situation in the Islamic Republic of Iran.

Content Link to heading

Section 1 introduces the Yazidi (Êzidî being the preferred term) people and their strong connection to their cultural heritage.

Section 2 provides an overview of the genocide against the Yazidis. The Islamic State made no secret of its intention to eradicate the Yazidi community and commenced a coldly calculated policy of ethnic cleansing and genocide on 3 August 2014.

Section 3 analyzes the relevant framework of international criminal law. The destruction of tangible cultural heritage is most easily prosecuted as a war crime. Several convictions were obtained before the ICTY, as well as one conviction (Al-Mahdi) and one indictment (Ag Mahmoud) before the International Criminal Court. Attacks against tangible heritage may also be prosecuted as the crime of persecution, a crime against humanity. Numerous indictments and convictions before the International Criminal Tribunal for the Former Yugoslavia (ICTY) and other courts attest to the viability of this approach. Finally, destruction of tangible heritage also serves as evidence of the special intent to destroy (dolus specialis) a protected group as part of the crime of genocide.

Section 4 provides original research, evidence and context on the destruction of Yazidi tangible cultural heritage in the Bahzani/Bashiqa and Sinjar areas of northern Iraq. We present satellite imagery analysis conducted by the EAMENA Project, drawing on data provided by Yazidi representatives. According to the Department of Yazidi Affairs in the Ministry of Awqaf and Religious Affairs in the Kurdistan Regional Government 68 Yazidi sites were destroyed by the Islamic State. We consider 16 sites in the Bahzani/Bashiqa area and 8 in the Sinjar area to which access was possible and which could be documented. We include description and religious importance of each site, satellite analysis and photographic evidence.

Section 5 offers conclusions and recommendations. We conclude that the destruction of the cultural heritage of the Yazidi people constitutes a war crime, a crime against humanity (persecution) and further evidence of genocide.

About RASHID International Link to heading

Info

All RASHID International research is available open access: zenodo.org/communities/rashid-international/

About Yazda Link to heading

About EAMENA Link to heading

[New Article] Heritage and Cultural Healing in Iraq

Fri, 26 Apr 2019 00:00:00 +0000

Info

[Open Access] Read the full article: doi.org/10.1080/13527258.2019.1608585

Abstract Link to heading

Against the backdrop of the destruction of Iraqi heritage over the past quarter of a century, this article critically reviews key aspects of the current state of Iraq’s cultural heritage, including damage to heritage buildings caused by Daesh in Iraq’s second largest city, Mosul.

We bring together Iraqi and non-Iraqi expertise in heritage, archaeology, and human rights law to frame our approach, building on the movement to link cultural diversity, heritage, and cultural rights. We emphasise the need for planning to enhance protection of Iraq’s heritage, in particular through the preparation of inventories, the provision of resources for heritage education in schools and the development of Iraq’s museum sector. Iraq’s presence on the UNESCO World Heritage Lists needs to be enhanced, and the issues of illicit site looting and traffic in looted antiquities must be addressed within international contexts.

Iraq’s future accession as State Party to the 1999 Second Protocol to the 1954 Hague Convention is a priority in achieving these goals. The paper stresses the need for co-creation of heritage knowledge and a gender-sensitive human rights approach for the future of Iraq’s globally significant cultural heritage.

About RASHID International Link to heading

Info

All RASHID International research is available open access: zenodo.org/communities/rashid-international/

[New Article] Enhanced Protection Policy Brief

Wed, 24 Apr 2019 00:00:00 +0000

Info

[Open Access] Read the full article: doi.org/10.5281/zenodo.3822201

Abstract Link to heading

Cultural heritage around the world continues to be threatened by war. The 1954 Convention for the Protection of Cultural Property in the Event of Armed Conflict (‘1954 Hague Convention’) automatically confers protected status on every cultural property “of great importance to the cultural heritage of every people”. This very universality, however, is also its greatest weakness: it does not provide visibility to individual cultural sites or help military planners and field commanders choose priorities.

In addition, the system of Special Protection under the 1954 Hague Convention is in a state of relative dormancy due to its requirement that sites be located an adequate distance from military objectives, and because of the politicization of the inscription process during the Angkor debacle of the 1970s. World Heritage status functions as a stop-gap measure for some sites, but is financially infeasible for most States and, in any case, provides no additional legal protection.

The system of Enhanced Protection under the 1999 Second Protocol to the 1954 Hague Convention addresses these shortcomings by uniting three important considerations: wider scope, ease of access, and strong legal protection. This article recommends that States ratify the 1999 Second Protocol, nominate relevant sites and objects for Enhanced Protection, and support the system of Enhanced Protection by providing and requesting financial and technical assistance from the Fund for the Protection of Cultural Property in the Event of Armed Conflict.

About RASHID International Link to heading

Info

All RASHID International research is available open access: zenodo.org/communities/rashid-international/

Posts on Seán Fobbe

[Essay] Reflections on Academic Communication

Overview Link to heading

Table of Contents Link to heading

Website and Blogging Link to heading

Essays Link to heading

RSS Feed Link to heading

Gatekeeping Link to heading

Asynchronous Publishing Link to heading

Speed Link to heading

Analytics Link to heading

Social Media Link to heading

Mastodon Link to heading

Linkedin Link to heading

E-Mail Link to heading

200,000 Legal Dataset Downloads

Thank you! Link to heading

Zenodo Download Statistics Link to heading

Applications Link to heading

[12 September 2025] Upcoming Agora Talk at the European Society of International Law (ESIL) Annual Conference in Berlin

Upcoming Talk on New Methods in International Law Link to heading

Timetable Link to heading

The Chains of Pride (Abstract) Link to heading

Slides for 2025 ESELS Toulouse Conference Talks

Slides for 2025 ESELS Conference Link to heading

[19-20 June] Upcoming Talks at the European Society for Empirical Legal Studies (ESELS) Conference in Toulouse

Upcoming Talks at European Society for Empirical Legal Studies (ESELS) Annual Conference Link to heading

Schedule Link to heading

UN Security Council (UNSC) Talk Link to heading

Federal Court of Justice (BGH) Talk Link to heading

[28 May 2025] Upcoming Talk on Legal Data Engineering (Online)

[New Pre-Print] Words of Power: Introducing a Comprehensive Corpus of UN Security Council Resolutions

New Pre-Print on UN Security Council Data Link to heading

Author Websites Link to heading

Migrating my Open Source Projects to Codeberg

Open Source Development now on Codeberg Link to heading

Why Codeberg? Link to heading

Notes on the Migration Link to heading

The Censorship of Science in the United States

[Essay] The Intelligent AI Coin: A Thought Experiment

Overview Link to heading

Table of Contents Link to heading

The Rise of LLM-based Decisionmaking and Agentic AI Link to heading

Freedom through Machine Intelligence? Link to heading

How Would You Even Know If Your AI Is Intelligent? Link to heading

The Intelligent AI Coin Thought Experiment Link to heading

Option 1: Evalute the Coins Yourself Link to heading

Option 2: Trust the Expert Link to heading

Some Objections Link to heading

Objection: Restrict AI to Low Value Decisions Link to heading

Objection: Make AI Open Source Link to heading

Objection: Add a Human to the Loop Link to heading

Objection: Human Agents are Untrustworthy Link to heading

Conclusion Link to heading

References Link to heading

A Better Social Media Alternative: Come Join us on Mastodon!

The Time for Open Social Media is Now Link to heading

Why not Bluesky? Link to heading

[New Data] Citation Network of the German Federal Court of Finance (2010-2024)

Overview Link to heading

Basic Network Statistics Link to heading

Network Diagram for the First, Second and Third Senates Link to heading

Citation Network of the First Senate Link to heading

Citation Network of the Second Senate Link to heading

Citation Network of the Third Senate Link to heading

Technical Note Link to heading

Further Reading Link to heading

[Essay] The Limits of Reproducibility with Rocker Docker Images for R

Overview Link to heading

Table of Contents Link to heading

Reproducible Scholarship Link to heading

The Software Stack in R Projects Link to heading

Docker Environments and Rocker Images Link to heading

The Limits of the Rocker R Package Version Freeze Link to heading

Solutions to Ensure Date-Lock of R Packages Link to heading

Solution 1: Use Non-Latest R version Link to heading

Solution 2: Pull Images Often Link to heading

Solution 3: Set CRAN Mirror Date Explicitly Link to heading

Tracing the Error Link to heading

Conclusion Link to heading