Public data is everywhere, collected by government agencies, advocacy groups, news organizations and an ever expanding set of specialty sites. It's online so many different places, it's impossible to search everywhere at once.
The accountability project is an effort to cut across these data siloes and give journalists, policy professionals, activists, and the public at large a simple way to search across huge volumes of public data about people and organizations.
The promise of a massive trove of government data released after the Obama administration's open data directive has mostly proved to be a mirage. The good-faith efforts a few government agencies have been mostly outweighed by the callous disregard of the order by others--like the Internal Revenue Service, or the U.S. Senate--with long histories of thumbing their noses at electronic data, a practice that costs taxpayers millions of dollars in paper record conversions and the legal fees of advocates who've beaten the agencies in court. Today, data.gov has thousands of datasets about fish, but none about the federal lobbyists fighting fishing regulation.
At the state and local level, the situation is worse. Data is kept in many different formats, often using technology that makes it difficult for the public to access. While some states and local agencies have made efforts to put data online, some of the most important accountability data is not available to the public or in difficult-to-manage formats.
We've standardized public data on a few key fields by thinking of each dataset row as a transaction. We're most interested in parties to the transaction, date, and amount of money involved. The name search can really be thought of as a search for parties we identified in a public record. Each party may include a name, either for a person or a business, as well as an address. The name search returns the number of name address combinations that match the search terms, and the dataset they appear in.
Clicking on the results found on the name search page will bring you to the dataset search page. The dataset search results shows all of the standardized fields from a row that a transaction party was involved in. The dataset search also shows all individual rows, whereas the name search doesn't display this level of detail.
We can fight this together! We've curated several hundred datasets we think are most relevant--but thousands more are just out of reach. If you know of a dataset that you think belongs here, suggest it for inclusion. We're especially interested in the data that agencies have hidden behind "search portals" or state legislative exemptions. Have you scraped a gnarly records site? Share it with us and we'll credit you. And more importantly, other people may benefit from your hard work.