Codebook

CLARA: Communication of Local German Party Associations, 2015–2025 · Version 1.0.0
← back to the dataset page

Codebook

This codebook documents CLARA (Communication of Local German Party Associations, Recorded Annually). CLARA records what local party chapters put on their own websites, so it is a panel of observable local party web presence rather than a census of local party activity, personnel, or issue positions. The unit of observation is the county-level party association, the Kreisverband, which is a party's association for one county (Kreis). Kreisverbände account for 2,591 of the 2,635 chapters and for 2,316 of the 2,319 chapters in the recommended analysis scope, which is the analysis_scope_included filter defined in the roster below. The small remainder are city- or town-level chapters, identified by org_level. A Kreisverband typically contains subordinate town-level chapters; those are not separate observations. Every substantive field is derived from website content, so a value exists only where the chapter had a website and the Wayback Machine archived it in that year. Condition analyses on observability_status, which is defined with the panel below (see also the data report and the coverage tables).

This release contains nine files, with identifiers stable across all of them:

chapter_roster.csv (one row per chapter)

Column Type Description
chapter_id integer Stable chapter identifier. Primary key; joins to the panel.
party string One of CDU, CSU, SPD, GRUENE, LINKE, AFD, FDP.
org_level string Organizational level: kreisverband, the county-level association (2,591 of 2,635 chapters; 2,316 of 2,319 in the analysis scope); ortsverband, a town- or locality-level chapter (36); stadtverband, a city-level chapter (5); bezirksverband, a district association covering several counties, one level above the Kreis (3).
org_name string The chapter name as it was discovered. Blank where the discovered value was a fragment of a URL or email address rather than a name.
url string Chapter website URL used for crawling.
url_source string How the chapter's URL was found. party_directory (listed in an official party directory) and url_probe (a predictable domain pattern that resolved) are high trust. spd_pdf_roster comes from the SPD's own published roster of chapters. serper_search, a web search whose results a language model matched to the chapter, is lower trust. archived_directory (3 chapters) means the URL was recovered from archived Wayback copies of the official party directories, which is how sites already offline when discovery ran could still be found. Method details and per-channel counts are in the README.
ags string The official 8-digit municipality key (Amtlicher Gemeindeschlüssel) of the chapter's seat municipality, the town the chapter is based in. Its digits run 2 for the federal state, 3 for the county, and 3 for the municipality, so kreis_id is its first five digits. Use ags for the seat town itself, either to place the chapter on a map or to merge town-level covariates. For anything measured at the county level, merge on kreis_id instead, because the chapter organizes the whole county rather than the seat town. Read as string: it is zero-padded and loses its leading zero (Länder 01–09, ~80% of chapters) under default integer inference.
municipality string Name of the seat municipality. This is the one town matched to the chapter, not the set of municipalities in the county the chapter covers.
kreis string County (Kreis) name. Do not join on it: county and city names collide, since a Landkreis and a city can carry the same name. Use kreis_id for joins.
kreis_id string The 5-digit county key (Kreisschlüssel), equal to the first five digits of ags. Because chapters are county-level associations, this is the recommended merge key for county-level covariates such as election results, demographics, or unemployment. Read as string; like ags it is zero-padded.
bundesland string Federal state.
population integer Population of the seat municipality (Destatis). For a kreisverband chapter this is not the population the chapter organizes: the chapter covers the whole county, and the seat town is one municipality inside it. Merge county-level population on kreis_id when the analysis needs the county figure.
is_quarantined 0/1 1 if the chapter failed a roster-integrity check: it sits outside the target roster (e.g. a state- or region-level association rather than a county one), carries a malformed name, or has an unusable URL. 225 chapters.
analysis_scope_included 0/1 The recommended analytic filter. It is 1 when the chapter belongs to the sampling frame and was not removed by the quality screening recorded in is_quarantined. The sampling frame is the set of 2,394 party × Kreis cells described in the README: 399 Kreise from the Destatis Gemeindeverzeichnis (built from municipalities with population ≥ 5,000), crossed with AfD, FDP, Grüne, Linke, and SPD everywhere, CDU outside Bavaria, and CSU in Bavaria.
n_usable_years integer How many of the chapter's years have observability_status = extractable_and_usable. Ranges 0–11, since the panel spans the eleven years 2015–2025.

chapter_year_panel.csv (one row per chapter × year)

The panel is rectangular: every one of the 2,635 chapters has a row in every year from 2015 to 2025, including years in which nothing was observed. A missing chapter-year in an analysis is therefore always a filter you applied rather than a row the release omits. Identifiers and geography (chapter_id, party, org_level, org_name, ags, municipality, kreis, kreis_id, bundesland, population) repeat the roster values on every row, so the panel can be used without joining the roster. That includes kreis_id, the recommended merge key for county-level covariates.

Observability

Column Description
observability_status Whether the chapter-year is usable and, if not, at which step it failed. Values include extractable_and_usable (enough archived content was fetched and parsed to trust the features), no_archive_coverage (no Wayback snapshot of the site exists from that year), crawl_succeeded_but_no_extractable_pages (snapshots were fetched, but none was a page type the extractor reads), crawl_succeeded_but_below_usable_threshold (pages were parsed but fell short of the minimum-content threshold), crawl_not_attempted (snapshots exist in the archive but were never fetched), and no_known_website (no URL was ever found for the chapter).
has_wayback_coverage 1 if at least one Wayback snapshot of the chapter's site exists from the same year. This is an exact recode of observability_status, not separate information: it is 0 for exactly the two statuses no_archive_coverage and no_known_website.
usable_content_threshold_met 1 if and only if observability_status is extractable_and_usable. Also an exact recode.
is_quarantined 1 if the chapter failed the roster-integrity check. Defined at the chapter level, so it is constant across a chapter's eleven rows (see the roster).
analysis_scope_included 1 if the chapter is in the sampling frame and was not quarantined; the recommended analytic filter. Also chapter-level, so constant across a chapter's years.

Because has_wayback_coverage and usable_content_threshold_met are recodes of observability_status, they carry no information it does not already carry. Do not enter them in a model as separate covariates alongside it.

Release coverage taxonomy

Column Description
coverage_basis Whether the release treats the chapter-year as researcher-usable, and on what evidence. full: the chapter-year clears the structured-coverage threshold, meaning at least 3 same-year pages were successfully fetched and at least 1 of them is of a page type the extractor reads (a board, council-group, organizational-profile, contact, issue, or news page). feature_level: it does not clear that threshold, but individual feature values were still observed, either on pages that fell below it or on pages a fallback rule routed to the extractor. Blank: nothing researcher-usable was recovered, or the row lies outside the analysis scope (analysis_scope_included = 0), where this column is not populated. Together full and feature_level give the release denominator, 17,989 chapter-years, or 70.5% of the analysis scope.
release_coverage_bucket The finest coverage classification. It names the specific reason for a chapter-year's coverage status (e.g. usable_coverage, no_archived_snapshot, feature_observed_below_global_threshold, archived_but_thin_content). Every bucket is defined in coverage_and_quality/coverage_taxonomy_summary.md.
release_coverage_tier A coarser rollup of the buckets, used for tables and figures: usable, limited_usable, weak_unlikely, missing_archive, archive_or_crawl_problem, missing_url, excluded.

For most purposes, filter first on analysis_scope_included = 1 and then pick one of two coverage rules. The conservative rule keeps only coverage_basis = 'full'. The inclusive rule keeps every non-blank coverage_basis, which is the full researcher-usable denominator. Running an analysis under both rules, with coverage_basis as the split, is a natural robustness check.

Features and their status columns

Every feature X in this file comes with a companion column X_status explaining why the feature has the value it has. observed_positive: extraction ran and found a positive value. observed_zero: extraction ran and found none, so the zero is a measured zero rather than a blank. coverage_available_not_extracted: pages that could have carried the feature were archived, but they were never parsed, so the gap is in processing rather than in the archive. missing_insufficient_coverage: there was no usable archived coverage to extract from in the first place. A feature value is informative only under the first two statuses; under the other two the value is missing, not zero.

Feature Type Description
board_listed 0/1 1 if the site lists a leadership board.
official_count integer Number of named officials extracted from the chapter's pages.
issue_block_count integer Number of issue content blocks extracted. It counts blocks of content, not distinct topics, and no cap is applied per page, so a chapter with one long issues page can score high without covering many topics.
office_address_listed 0/1 1 if the site lists an address for a local party office.
facebook_present 0/1 A chapter-specific Facebook account is present (national party accounts excluded).
instagram_present 0/1 A chapter-specific Instagram account is present (national party accounts excluded).
twitter_x_present 0/1 A chapter-specific X/Twitter account is present (national accounts excluded).
youtube_present 0/1 A chapter-specific YouTube channel is present (national accounts excluded).
telegram_present 0/1 A chapter-specific Telegram channel is present (national accounts excluded).
tiktok_present 0/1 A chapter-specific TikTok account is present (national accounts excluded).
n_social_platforms integer Count of chapter-specific platforms present among the six above (0–6).
social_source text Which of two detection paths measured the social platform flags for this chapter-year. about_extraction: a language model read the chapter's about or contact page. page_scan: no about or contact page was available, so the stored page markdown was scanned for platform URLs by a deterministic rule (see coverage_and_quality/missingness_audit_addendum.md). Null when social presence was not measured at all.
news_count integer Deduplicated news articles assigned to this chapter-year (by publication-date year where a date could be extracted, by snapshot year otherwise). It reflects how densely the Wayback Machine archived the site as well as how much the chapter posted; see chapter_year_news.csv for label shares and n_news_pages (enriched panel) for the raw archived-page proxy.

Interpreting the panel

chapter_year_panel_enriched.csv (core features + derived measures)

This file has one row per chapter × year, the same rows as chapter_year_panel.csv. It keeps the panel's columns, per-feature status columns included, apart from the redundant recode has_wayback_coverage, and adds derived measures computed from the underlying extracted records. Those derived measures carry four status columns of their own: issues_present_status, chair_gender_status, chapter_head_gender_status, and web_activity_status. Each is documented below alongside the measure it governs. This file is therefore the one to use for missingness-aware work on the derived measures, not just on the core features. Every value is an aggregate: no names, emails, or text appear anywhere in it. A measure is null when the records it is built from were not observed for that chapter-year.

Board composition and professionalization

Column Description
n_female, n_male How many of the extracted officials had a first name classified as female / male.
female_share n_female / (n_female + n_male): the share of women among the officials whose gender could be classified. Officials whose name could not be classified drop out of both the numerator and the denominator.
gender_coverage Share of the extracted officials whose gender could be classified at all. When this is low, female_share rests on a small and possibly selected part of the board, so read it cautiously.
chair_present 1 if any official is categorized as chair (Vorsitzende/r).
chair_female 1 if a chair is female, 0 if a chair is present but not female, null if no chair.
chair_gender_status Why chair_female has the value it has. officials_unobserved: no officials were observed for the chapter-year. no_chair_identified: officials were observed but none was categorized as chair. chair_gender_unresolved: a chair was identified but no chair's gender could be classified. observed: a chair was identified and gendered.
chapter_head_present The strict leadership measure. 1 if at least one person was identified as a current substantive head or co-head of this chapter itself. 0 if every extracted official was successfully classified and none of them heads this chapter. Null if at least one official could not be classified either way, so the answer is unresolved rather than negative. "Focal chapter" throughout means the chapter the row is about, as distinct from a parliamentary faction, an affiliated group, or a different organizational level.
chapter_head_female 1 if any confirmed head of the focal chapter is female. 0 only when every confirmed head was resolved as male and no unresolved candidate could overturn that. Null otherwise, including when a head is confirmed but their gender could not be resolved.
n_chapter_heads Number of confirmed current heads or co-heads of the focal chapter.
n_chapter_heads_unknown_gender How many of those confirmed heads have unresolved gender.
n_unresolved_head_candidates Number of extracted officials for whom the classifier could not settle whether they head the focal chapter. A count above 0 is the reason chapter_head_present can be null.
chapter_head_gender_status Why chapter_head_female has the value it has. officials_unobserved: no officials were observed for the chapter-year. head_identification_unresolved: officials were observed, but it could not be settled which of them heads the chapter. no_head_identified: officials were resolved and none heads the chapter. head_gender_unresolved: a head was identified but their gender could not be classified. observed: a head was identified and gendered.
academic_title_share Share of officials listed with a Dr. or Prof. title.
email_share Share of officials with a listed email.
n_distinct_role_categories Number of distinct board functions named among the officials, counting specific functions only and ignoring generic "member" or "other" labels.
photo_share Share of listed officials whose entry includes a photo.
n_candidates Number of officials categorized as electoral candidates (e.g. on candidate-list pages).
board_completeness 1 if the board lists all three core functions (chair, treasurer, secretary), else 0. Larger boards clear this bar more easily, and it overlaps chair_present and n_distinct_role_categories, so do not treat the three as independent measures.
officials_source Which kind of page supplied the board records: party_office (the chapter's internal officers), representative (elected council members or the party's Fraktion, its group in an elected council), mixed, or other. The two page types list different people, so condition board and gender measures on this column; see the caveats below.

Issue emphasis

Column Description
issues_present_status Whether issue content was observed for the chapter-year, using the same four values as the panel's feature statuses. The three columns below, and the issue-topic presence flags, are populated only under observed_positive. observed_zero means extraction ran and found no issue content, so the absence is measured rather than a gap in coverage.
n_topic_families Number of distinct policy families the chapter-year's issue blocks fall into (the block count itself is issue_block_count in the panel).
dominant_topic_family The policy family holding the most issue blocks that year (e.g. environment, governance, education). It is a summary of chapter_year_topic.csv and adds no information beyond it.
topic_diversity Normalized entropy of the issue blocks across policy families, rescaled so 0 means every block sits in one family and 1 means they are spread evenly. Because the rescaling is relative to the families actually observed, two evenly split blocks already give 1.0; read it alongside issue_block_count.

These fields measure issue salience, meaning which topics a chapter writes about at all. They do not measure position or stance: whether a chapter is for or against something is not coded.

The long file chapter_year_topic.csv (chapter_id, year, topic_family, n, cap_major_topic) gives the full per-family issue-block counts for all 26 families. Its cap_major_topic column maps each family to a Comparative Agendas Project (CAP) major topic (see topic_family_cap_crosswalk.csv) so the data can be used alongside CAP-coded sources. The mapping is best-fit, and a few families (tourism, sport, family) involve documented judgment calls.

Issue-topic presence flags

Wide 0/1 flags for the 15 most common policy families, derived from the same topic classification as the counts above. topic_<family>_present = 1 if the chapter-year has at least one issue block in that family, 0 if issue content was observed but none of it fell in that family, and null if no issue content was observed at all. Families: governance, economy, environment, education, mobility, social, urban_development, culture, finance, health, integration, housing, digital, security, elections. Per-family counts for these and the remaining families are in chapter_year_topic.csv.

Web activity and mobilization

These are structural signals read off the archived pages by deterministic rules, with no page text stored.

Column Description
web_activity_status Whether any archived source page was seen for the chapter-year. observed: at least one, so the columns below carry values. no_source_page_observed: none, so every column below is null.
n_pages_total Number of successfully archived pages captured for the chapter-year.
n_distinct_page_types Number of distinct page functions among those pages (homepage, news, officials, and so on). It proxies how broad the site is.
n_news_pages Number of archived news-type pages. It proxies posting activity but also reflects Wayback archival density (correlated ~0.9 with n_pages_total); use it conditional on observability_status.
donation_link_present 1 if any archived page exposes a donation pathway (e.g. "Spenden").
join_party_link_present 1 if any archived page exposes a membership/join pathway (e.g. "Mitglied werden").
newsletter_signup_present 1 if any archived page references a newsletter.
contact_form_present 1 if any archived page references a contact form.
program_present 1 if any archived page references a party/election program (Wahlprogramm, Grundsatzprogramm).
event_calendar_present 1 if any archived page references an events section (Veranstaltungen/Veranstaltungskalender).
volunteer_link_present 1 if any archived page exposes a get-involved/volunteer pathway (Mitmachen, Ehrenamt).

The seven link flags (donation_, join_party_, newsletter_, contact_form_, program_, event_calendar_, volunteer_) are keyword searches over the page markdown. Links of this kind usually sit in a site's persistent header or footer, so a positive flag means the pathway exists somewhere on the site, at the level of the site template. It does not mean the chapter was actively campaigning or fundraising that year. Positive rates are correspondingly high and vary by party. The event_calendar_present keywords exclude the ubiquitous "Termine", so that flag discriminates rather than saturating near 90%. These features operationalize the information-provision, participation, and mobilizing functions of Gibson & Ward's (2000) party-website scheme. Flags are null where no page was archived for the chapter-year. Use them, along with n_news_pages and n_pages_total, conditional on observability_status.

Year-to-year reliability

Feature values are measured from whatever the Internet Archive captured of a site that year, and capture varies from year to year for the same chapter. This adds noise to comparisons of a chapter with itself over time, but not to comparisons across chapters. I measure it two ways, both computed only where a feature's status column marks the value as informative.

For counts and shares I use the autocorrelation of first differences. A persistent quantity measured without error gives about 0, noise as large as the true annual changes gives about −0.33, and pure noise gives −0.5. The release values are −0.42 for issue_block_count, −0.42 for official_count, −0.31 for n_social_platforms, and −0.25 for news_count. Computed by party, the same statistic ranges from −0.18 to −0.50 for issue_block_count, so cross-party comparisons of within-chapter dynamics carry unequal amounts of noise.

For binary flags I count how often a value differs from both its neighbors while the neighbors agree, divided by how often the value changes at all. Independent noise gives 0.5, a clean one-time adoption gives 0. The issue-topic presence flags run 0.40 to 0.44. The social platform flags run 0.12 to 0.31, so their changes are mostly real adoption. The flags in between behave as though roughly one chapter-year in ten is misclassified.

Part of this noise is systematic rather than random. The year-to-year change in issue_block_count correlates 0.28 with the change in log page count, and official_count 0.17. In a chapter fixed-effects regression, adding log page count cuts the annual trend in issue_block_count by about three quarters, turns a flat trend in official_count into a declining one, and leaves n_social_platforms almost unchanged. Page counts partly reflect real activity, so this control can absorb signal along with noise. Report estimates with and without it and read the difference as a bound.

Noise of this kind costs precision when a measure is the outcome and attenuates the coefficient when it is a regressor, with differencing making the attenuation worse. Averaging over multiple years helps issue_block_count and n_social_platforms but does little for official_count, so it is not a general remedy.

Method and caveats

Relation to standard coding schemes

The feature set maps onto established website- and text-coding frameworks. Gibson & Ward's (2000) five party-website functions (information provision, resource generation, networking, participation, campaigning) correspond to the program/contact, donation/join, social-link, event/volunteer, and news/activity fields. Issue families crosswalk to Comparative Agendas Project (CAP) major topics via topic_family_cap_crosswalk.csv. Issue fields measure salience, not position.

chapter_year_news.csv (one row per chapter-year with archived articles)

News volume and content-label shares per chapter-year, built from the deduplicated article records. Every article was classified by the same locally hosted open-weight model into a topic family, an article type, and a geographic focus; the human validation of those labels is reported in the quality section of the README. The file holds one row per chapter-year with at least one archived article, 14,733 rows in all. A chapter-year absent from this file had no article to summarize.

Column Description
news_count Deduplicated articles assigned to this chapter-year (same value as the panel column).
n_dated Articles whose year comes from an extracted publication date (the rest use the snapshot year).
n_labeled Articles with content labels (denominator for the share columns).
share_local / share_regional / share_national Share of labeled articles by the geographic focus of their content; the remainder to 1 is unclear.
share_press_release / share_event / share_position / share_campaign / share_council Share of labeled articles by function: press release, event announcement or report, programmatic position, election campaigning, council and Fraktion work. The remainder to 1 is other.

The share columns describe the composition of what a chapter posted and the archive captured, not how much it posted; a chapter-year with three archived articles can show a share of 1.0. Combine the shares with news_count whenever volume matters. The campaign share tracks the electoral calendar, peaking in 2017, 2019, 2021, 2024, and 2025.

chapter_year_news_topic.csv (long format, news topics)

Per-family article counts per chapter-year (90,668 rows), mirroring chapter_year_topic.csv but built from classified news articles: chapter_id, year, topic_family, n, cap_major_topic. Issue-page topics come from static program content and change little within a chapter, whereas news topics move year to year, so this file is the one to use for issue-attention dynamics. Articles labeled other or unclear are excluded.

chapter_year_provenance.csv (one row per chapter-year)

Source-level provenance for each chapter-year, recorded so that any aggregate in the release can be audited without publishing page text or personal data. Keyed on chapter_id and year.

Column Description
n_snapshots Wayback snapshots available for the chapter-year, counted from the CDX index, the Wayback Machine's catalog of archived URLs.
unique_digests Distinct content digests among those snapshots, that is, how many genuinely different versions of the pages the archive holds.
capture_first, capture_last Earliest/latest Wayback capture timestamps (YYYYMMDDhhmmss); read as strings.
capture_year The panel year.
n_source_pages Successfully archived pages with content (equals n_pages_total in the enriched panel).
n_distinct_page_types Distinct page functions among them.
n_officials_pages, n_news_pages, n_issue_pages, n_about_contact_pages Successful pages by function.
selection_mode Which rule chose the snapshots to fetch for this chapter-year. structural_priority picks archived URLs by page function, taking structural pages such as officials and contact first, then issue and program pages, then news, then everything else, instead of applying one flat page cap. Blank: the chapter-year's pages were fetched in crawl rounds that applied a flat page cap, before this per-round record was kept.
selected_total Snapshots selected for fetching under that rule.
url_cap_hit 1 if the per-chapter-year URL cap bound the selection, meaning further archived URLs were available but not fetched.
archive_hint_urls_added Extra archived URLs fetched beyond those discovered on the live site. These come from the archive itself and partly offset survivorship bias, since a page that disappeared before discovery ran can still be present in the archive.
n_extraction_attempts, n_extraction_ok Extraction attempts and successes over those pages.

Use these to gauge how much source material underlies a chapter-year's measures and to condition on capture timing.

kreis_wahlkreis_crosswalk.csv (one row per county × election × district)

Counties and Bundestag electoral districts (Wahlkreise) do not nest. For the 2025 election 293 counties lie entirely inside one district while 107 span two or more, and a district can equally cover several counties. District-level election results therefore cannot be joined onto the panel by county alone. This file supplies the weights that make the merge possible for the 2017, 2021, and 2025 federal elections. For each county and election it lists every district the county falls in, together with the share of the county's population living in the part of the county assigned to that district. Join on kreis_id and the election you need, then aggregate district results using population_share as the weight. The file has 1,640 rows and is unique on county, election year, and district.

Column Type Description
kreis_id string The 5-digit county key, matching kreis_id in the roster and in both panel files. Read as string; it is zero-padded.
election_year integer The federal election the assignment applies to: 2017, 2021, or 2025. District boundaries were redrawn between these elections, so a county's set of districts can differ from one election to the next; it does so for 250 of the 401 counties in the file.
wahlkreis_nr integer Official number of the electoral district, 1 to 299.
wahlkreis_name string Official name of that district.
population_share float Share of the county's population living in municipalities assigned to this district. Within a county and election year the shares sum to 1, up to rounding in the published values. This is the weight to use when aggregating district-level results to the county.
n_municipalities integer How many of the county's municipalities are assigned to this district.

The assignments come from the Bundeswahlleiterin's official municipality-to-district lists for each election, combined with municipality populations taken from the edition of the Destatis municipality register closest to that election's territorial status.

The weights come with two limits. First, where a single municipality is itself split across several districts, the published assignment gives no sub-municipal population figures, so that municipality's population is divided equally among the districts it touches. This affects 18 large cities, and for them population_share is an approximation rather than a measurement; in the file it shows up as exactly equal shares across a county's districts, as for Köln in 2025, which carries 0.25 in each of its four districts. Second, the file covers every German county, which is a superset of the 399 counties in the sampling frame. It also carries Cochem-Zell, which has no municipality of at least 5,000 residents and therefore falls outside the frame, and Eisenach, which was a county in its own right for the 2017 and 2021 elections before being merged into the Wartburgkreis. Rows for counties outside the frame find no match when the file is joined to the panel.