To: NYC Council – Committee on Technology
From: Noel Hidalgo, Executive Director of BetaNYC
Re: Open Data Hearing
Tuesday, 27 February 2024
Dear Chair Gutiérrez, fellow Council Members, and Staff,
Now, more than ever, the City’s data analytics team and the open data program need resources to ensure other agencies can do more with less. Since its inception, the data analytics team has streamlined internal and external data-sharing practices. Their work has improved people’s lives.
We’re a quarter of the way through the 21st Century, on the verge of a climate catastrophe, and need to ensure that the human investments we make at this moment will carry us through the middle part of the century. Consumer technology systems will morph rapidly, and the climate will dictate swift responses to people’s needs. In sum, we need adaptable people who will be around for a bit.
Additionally, artificial intelligence and machine learning tools require good, clean metadata. We can NOT build a 21st-century government on the back of systems that contain poor data. Management practices that paper over insufficient data should not be tolerated — people who are juking the stats should be held accountable.
If we’re talking about the City’s open data program, we need to invest in data analysts and the technologists and service designers they work with. These people can tell us the difference between spread-sheeeets from Shinola.
Opportunities and Insights:
Staffing — The open data team is down five additional people from pre-pandemic staffing levels. While the Mayor has lifted elements of the hiring freeze, we ask that the Open Data Team be restored to its pre-pandemic capacity and add three more to assist with civic engagement, data literacy, and engineering.
Cultivating New Leadership — Agencies must modernize data systems and update the civil service system to ensure we have systems that work for this decade, not the last one. We need government technical talent ready to understand new technologies and ensure that Agencies are not being sold snake oil. We need a new generation of technical leadership that understands the value of service design, data portability and can build upon open technologies. We must cultivate a new generation of technology leaders to do more with less. The analytics exchange, service design forum, and open data ambassadors are three programs demonstrating how technology/data/design leadership should cut across agencies. This must be combined with apprenticeship programs that prepare various college students to foster a career in public interest technology. Our Civic Innovation Apprenticeship programs can help model future career development paths.
Institutionalizing Open Data Coordinators and Open Data Ambassadors — We need people who know the ins and outs of datasets and can communicate that with others. We would like to see full-time Open Data Coordinators (internally) and Open Data Ambassadors (externally) within major agencies. ODCs and ODAs should be laser-focused on data quality and timeliness of publication; additionally, they are the right people to communicate the intricacies of datasets to broader audiences.
Accountability — As the Council is responsible for holding the Administration accountable, Commissioners and Agency heads must be asked about their internal infrastructure upgrades and agency data management practices at Budget hearings. Every Budget hearing should hold up whatever data is published in the dynamic Mayor’s Management Report, have it cross-referenced by data published in the open data portal, and then compare that to what they tell you. FOIL requests and responses should be included in these hearings. The Open Data Team should not be held responsible for what is out of their control.
Data Publishing — Several agencies do not have their act together. The Department of Education continues to be an outlier with nonstandard data publishing practices. Additionally, I do not understand why the NYPD continues to publish monthly Compstat data in Excel spreadsheets at the police precinct level while not publishing data directly to the open data portal.
Digital and Data Literacy — we’re on the verge of a transformative shift in how people interact with and incorporate information. If we are not teaching New Yorkers how to look at data or review source data, they will be prey to misinformation. Open Data Week, School of Data, Data Through Design, Open Data Ambassadors, and Mapping for Equity are just a few fundamental tools this government should adequately resource to increase data literacy while battling fake news and misinformation.
Thank you for giving us this opportunity.