Run the analysis locally with our open-source toolkit. Prepare a text file, execute the scorer (can use APIs from OpenAI, Google, or X), and share the results with full transparency.
Research basis: Delgado-Mohatar & Alelú-Paz, When Algorithms Guard Democracy — integrating Levitsky & Ziblatt’s four dimensions with LLM analysis.
Scores are derived from transparent prompts and rubric; single extreme utterances matter (maximum-score emphasis).
Create a plain text file with the full transcript of the speech.
Execute the toolkit locally. It can use APIs from OpenAI, Google, or X based on your configuration.
Inspect the CSV scores and generated charts, then share or audit the results.
The approach prioritizes early warnings by tracking maximum values per indicator; a single extreme statement can normalize anti-democratic behavior.
The toolkit alerts; it does not censor. It’s a public instrument for vigilance and accountability.
Prompts, indicators, and evaluation criteria are published so anyone can reproduce results.
The corpus and models are regularly updated to reflect new rhetoric and languages.
We highlight dataset limits, translation bias, and the risks of over-generalization across cultures and eras.
Follow these steps to execute the toolkit on your own machine.
Download our code from here.
Create a text file with any name and extension. For example: leader_X.txt.
Execute the scorer with:
# python3 evaluate.py --autor leader_X --speech_file leader_X.txt
When it finishes, output files will be in the evaluations/ directory.
The CSV file will contain the scores for each category.
Run the analysis with:
# python3 analysis.py --csv_file <<evaluations/csv_file>>
You will find a series of charts in the analysis_results/ directory.
python instead of python3.
The toolkit can use APIs from OpenAI, Google, or X based on your configuration.
We map language to four diagnostic dimensions: rejection of democratic rules, denial of opponents’ legitimacy, tolerance of violence, and readiness to restrict civil liberties. Results emphasize maximum indicator scores to capture extreme utterances.
Yes. We publish prompts, indicators, and evaluation criteria so others can replicate and critique findings.
No. This is a preventive, public-interest monitoring toolkit. It alerts, it does not censor.
Analyses depend on transcript quality, translation, and historical/cultural context. Numerical operationalization is a simplification and should be interpreted cautiously.
You can contact the authors at the following email addresses:
Please include the speech file name and a brief description of your request when contacting the authors.