Asheville’s Open Data Journey: Pragmatics, Policy, and Participation

Much of the open data conversation centers on policy, politics, or solving community problems. These are great—and needed—beginning and end points, but there is an important middle point that raises two questions: How do we produce open data? And how can we get open data to be a part of the government process?

Today, local, state, and federal governments generally have to go an extra mile to convert “open records” into what we would recognize as “open data.” This essentially means open records that are presented in a convenient, automated, and self-service format.

That extra mile is a tough nut. For all of the benefits of open data, this innovation also creates problems along the journey that make it easy for detractors to undermine open data efforts—think the benefits of social media versus Facebook addiction.

For example, legislators passing open records laws might not have thoroughly contemplated that these records would be generally available—instantly—in near-real-time to anyone who requests that data. Is it OK to distribute a list of everyone who holds a pistol permit? Open records laws in North Carolina, where I live, seemed to imply “yes,” but a local journalist got into fairly hot water when he planned to post the list and demanded that a sheriff provide these records in accordance with the law (Henderson and Steele, 2013).

What about the general freak out that government employees have when their salaries are published, in time-honored tradition, once a year in the local papers? Imagine the drama at any business when real-time salaries are publicly available for all employees. Someone inevitably ends up asking, “Why is Jane making a thousand dollars more a month than me?”

Change in general is hard, as those in IT and innovation understand very well. The seminal works on change—whether you’re talking Chip and Dan Heath’s Switch: How to Change Things When Change is Hard or John P. Kotter’s Leading Change—are no less relevant to the open data movement than any other innovation.

In Asheville, North Carolina, where I work, building an awareness of the need for change among multiple groups was a key strategy for creating an environment that was supportive of the change. Despite potential new problems that could be created by the presence of robust open data, the lack of open data created significant problems as well.

All municipalities in North Carolina must respond to open record requests per the North Carolina General Statute, Chapter 132. Citizens, businesses, and journalists may all want access to government records, and the traditional process is painful for everyone. Here’s how it typically goes:

1. The requester puts a request into a government department. As an illustrative example, let’s say that it’s business license data and that the requester contacts the city finance department.

2. In order for the request to be tracked and followed up on, the right place to contact is the public information office. (Many cities have found that when departments fulfill record requests on their own, the ball can sometimes get dropped—this is analogous to the modern day IT help desk, which serves to track service requests.) So, in our example, let’s assume that the person in the finance department is on the ball and forwards the request to the Public Information Office (PIO) with no further delay. That’s not always a great assumption.

3. IT or the government department pulls the requested data.

4. The city’s legal function reviews the data to ensure that the city isn’t breaking any laws in distributing the data. If certain data must be redacted (confidential, trade secret, secretary of state information, or other data specified in the law), they instruct IT to do so.

5. IT provides the final work product to the PIO, who then distributes it to the requester.

There are a lot of good reasons for this degree of scrutiny and time spent on the request, but the bottom line from my perspective is this: what if this was only done once for each dataset and future requests could be self-service? That was the primary pragmatic benefit we sought from our open data effort.

I am active in Asheville’s startup and entrepreneurial community, so I learned that Asheville was home to a company located downtown whose business model depends on access to government records. The company, BuildFax, employs thirty-five people in our community. I spent some time with their CTO and learned that they had gone to extraordinary pain and expense to integrate the disparate record data into their product offering—essentially, property condition analysis for the insurance and mortgage industries. They faced boxes of printouts that needed to be structured, machine-readable data. It was an interesting example of what entrepreneurs in our community might face if they wanted to build value out of the public data our government stewarded.

I also have witnessed firsthand the quiet aggravation that journalists have when faced with delays in record requests, even when they are legitimate delays.

All of these circumstances were occurring in an environment where finding speculative funding for new projects was difficult. The thought experiment goes like this: “Should we do a project that may reduce staff time involved in record request fulfillment, or should we replace an aging patrol car?” As you would expect, and as is appropriate, urgent public safety needs should and do come ahead of IT innovation. So, in a pretty tight budget, these projects don’t get funded.

There’s another problem: when data is not available to the public, this actually means that government IT or an expensive vendor must do everything themselves. Civic hackers can’t hack into data that they can’t access, so how can they innovate? These people could code a project as impactful as Linux, but they can’t even get started building for government.

Nobody wins.

In a world where nobody talks to one another about these problems, you could imagine the following insane scenario. Journalists and others complain to lawmakers about barriers to getting data from government. Lawmakers pass a burdensome law that cities have an obligation to publish a taxonomy of all data tables that are available. Journalists then ask for “everything,” even though the law prohibits them from receiving “everything” (social security numbers, personnel records, certain law enforcement data, etc.). Then, system vendors declare their data taxonomies (sometimes called “data dictionaries”) as proprietary and confidential information. Even when contracts are signed by vendors that refer to the state law, they require government IT workers to sign non-disclosure agreements if they want to gain access to the data tables. This is a truly insane rock and a hard place.

The problem is, this is actually how it often works (or doesn’t work) in North Carolina.

In a world where everybody talks to each other about these problems, you could well imagine that folks who need something would simply identify what they need and that the different groups would work together to make it happen.

In our community, after a brainstorming meet-up with others from diverse organizations (a local broadband provider, an entrepreneurship group, local businesses, government, the Chamber of Commerce, an IT networking group), a team of organizers formed and launched a conference called Open Data Day (opendataday.com). The idea for the day was this:

• It would be a “big push” to introduce the problems associated with a lack of open data to citizens, journalists, business people, elected officials, and others in the community and region.
• It would clarify that open data is more than just a government IT or civic-minded individual’s problem.
• It would feature national speakers from communities that had success stories surrounding the open data problem.
• It would act as a launch event for our city’s open data portal.
• It would feature a hackathon that would act as proof of the concept of what could be possible by using the data on the open data portal.
• Above all else, it would make the problem of open data into a community problem, not just a city government problem.

In fact, we did exactly that. We brought in Code for America’s Brigade director, Kevin Curry, as well as Robert Cheetham from Open Data Philly and Theresa Reno-Weber, who serves as Chief of Performance Improvement in Louisville, Kentucky. We launched a data portal. With ten community sponsors, twenty workshop speakers, and 130 attendees, it was a memorable event that created relationships and acted as a springboard for other open data and coding activities in the community (notably the formation of a Code for America Brigade—a group of volunteer civic hackers that meets on a regular basis).

I must emphasize that none of it would have happened if not for our community organizers. Because a diverse set of organizers were behind the event, it was perceived as a community event, rather than a city government event. That made a big difference in how it was accepted both at City Hall and by the community itself. (Who really wants to come listen to a city IT department spout off? Nobody, right?)

Many IT folks have a deep level of discomfort with citizen interaction. They leave it to the public information office, city council, or a neighborhood liaison. I don’t think that this can create good outcomes in your community any more than you would have a good outcome for an internal city IT project by avoiding any interaction with internal departments. Business technology exists to serve line departments, of course, but ultimately, business technology exists to serve citizens. It is ridiculous to think that you can have a good citizen outcome if business technology leaders avoid interaction with citizens. Of course, in a large community, you cannot expect that you’ll be able to interact with each citizen, so citizen groups are important.

Community organizers for our event came from citizen and business groups, and this contributed to momentum, publicity, attendance, and so on. One particularly important group—prior to the formation of a Code for America Brigade in our city—was the community’s entrepreneurship group, Venture Asheville, which provided sponsorship, as well as visibility and credibility in the tech startup and coder community. “Meet The Geeks,” a networking organization for IT people in town, was another important organizer group.

In terms of outcomes, the community’s open data day effort resulted in:

• Journalists being able to candidly speak about challenges that they had in getting data.
• A lower fear level among local government staff regarding the opening of data due to a greater understanding about the consumers of that data and a greater understanding that the benefits outweigh the risks.
• Great local press coverage, which further educated the community (Forbes, 2012).
• A successful hackathon, with the winner being an app that mashed up data from bus routes and public art to steer citizens to visit public art via bus.
• A platform for the launch of a Code for America Brigade called Code for Asheville.

Challenges and problems naturally remain. Open data shows up so much in the ideological world that city staff can still be worried that participating in open data efforts could be construed as political activity (political activity for city staff is generally not allowed). Also, despite education, there’s still confusion about the difference between an open record and open data. Technology and automation is never going to declare a record “open” if it is not open by law, but not every stakeholder understands that.

Sustainability

Our experience demonstrates that a “product launch” is an effective way to get the ideas about problems and solutions out into your community. It’s easy for big bangs to fizzle out, though, so an ongoing effort is needed, no matter how you begin open data efforts.

It is obviously not sustainable or practical to publish all datasets immediately. It could also undermine efforts if there is a perception that government staff, particularly IT staff, is arbitrarily or capriciously picking datasets to publish—that is, that the government is acting unfairly.

As a natural partnership between the public information office and IT has emerged, we’ve used a three-pronged approach for open data publishing criteria. This makes sure that the politics and policy stay with elected officials, the publishing choices city staff makes are reasonable, and we are using our time well.

Pragmatics

Does publishing a dataset save staff time? Does it lower the burden of public record requests by automating a frequently requested or time-intensive dataset? Does it decrease the cost of government? Does it make the process easier for citizens? Does it contribute to a business goal of a citizen-facing business unit (for example, police, fire, or development services, to name a few)?

Participation

This criterion asks how many votes a dataset has on the open data catalog site. If two people want it, it’s probably not a great candidate in a community of 85,000 citizens. We actually haven’t had a nominated dataset receive strong support from the community. When there is clear support for a dataset to be published, though, we’ll do our best to get it out there.

Policy

What does our governing board want? If our board says to publish a dataset that hasn’t met the other two criteria of “pragmatic” or “participation” to publish, let’s do it.

We feel that following these criteria take staff out of making policy decisions, while also not tying staff’s hands or delegating everything to a very busy governing board.

The pragmatics criterion is really important. Most cities, especially those with populations under 100,000, don’t have infinite resources. Creating self-service data where there is frequent time spent and creating opportunity for others to build on that data simply makes sense from an operational efficiency standpoint. Our data proves that open data saves IT, legal, and PIO time. We track unique events on the open data catalog (that is, we do not count if someone downloads the same dataset twice), and have seen pretty consistent growth from month to month. In 2013, we averaged 144 unique data catalog requests per month, which may not seem like a lot, but let’s put that in perspective. Our old way of handling these requests would generally take at least thirty minutes per individual (if not several hours), so by the time legal, PIO, possibly the department, and IT touched the request, we’re talking about two hours of fairly expensive and scarce staff time—at a minimum. Even if you were seriously conservative and assumed no legal or department involvement and just accounted for minimum levels of IT and PIO, that’s around seventy hours of work for PIO and IT per month.

Really? Almost two weeks of work? Sure, before open data, we probably didn’t have that much (indeed, early data showed about seventy catalog requests per month, meaning about a week of work if it was minimum and manual). We’ve made data more accessible, though, so folks no longer have a barrier to getting it. Therefore, more people are using it, without having to hire more staff to handle it. That’s an important outcome as well.

Next Steps

Once you launch a data portal, drum up interest, and create conversations, it’s my opinion that the right thing to do is back off a little bit. This is hard (at least it was for me) because, as an internal advocate for something that you believe in, you’ve put a lot of work and time into it.

Let me be clear: this is not about backing off because you will have detractors. Any innovation garners its share of detractors, and, sadly, the misinformed citizen’s “you-work-for-ME-personally” syndrome is alive and well amongst a vocal minority of citizens. Open data is no different. You will have detractors, and you will have to deal with them.

More importantly, though, there are others who have ideas and want to participate.

Don’t stop leading internally. It’s important to continue to communicate about the value of open data in the organization and continue to tell the story of how open data and self-service allows staff to focus on more meaningful work than manual data pulls.

Externally, when you create a little bit of a leadership vacuum by backing off just a little bit, you create opportunity. Natural leaders step up. In our community, a couple of government employees volunteer outside of work to help organize the Code for America Brigade (good for them, as well as the community), and more than a dozen non-government employees are also helping to organize, create code, arrange meet-ups, and so on. At least one university professor and one elected official stepped up into leadership roles in the 2013 National Day of Civic Hacking event in our city, called “Hack for Food.”

The “Hack for Food” event challenged folks to solve community challenges: a lack of healthy foods in schools, getting healthy food to people who lack access, and ensuring adequate food supply for the region in case of a crisis. The publicity for the event did talk about open data, hacking, and code—all of which were useful tools—but it primarily focused on solving community problems.

The organizers chose a community priority that had previous governmental and community action surrounding it. For example, the Asheville-Buncombe Food Policy Council’s mission is to “identify and propose innovative solutions to improve local food systems, spurring local economic development and making food systems environmentally sustainable and socially just.” Also, Asheville’s city council passed resolution No. 13-17 on January 22, 2013, that supports municipal food policy goals and an action plan around those goals.

The point: no matter what the community priority is, focusing on the priority and not just the tools is helpful in getting folks excited. When the leadership comes from the community, you can bet that something good will happen.

As the community starts to rally, I think that the best place for the internal open data advocate is as a bridge-builder or interpreter. After all, that open data advocate understands both worlds.

For example, not everything fits neatly into the pragmatics-participation-policy filter. When it doesn’t, a little bit of bridging can help. In one case, Asheville’s Brigade redeployed an open source budget transparency application called Look at Cook. This app is exactly the type of budget transparency app that ambitious government IT folks have been dreaming about for a decade, but could never fund when faced with other needs. It is squarely in the “want to” quadrant, and government IT spends most of their time in the “must do” and “should do” quadrants. The point is, assuming good intentions from citizens and the government; it’s something that everybody wants.

The data for the app was not immediately available (and it was not immediately obvious that this dataset fit one of the criteria), but it was an open record, so the Brigade put in an open records request. The budget office was more than willing to provide data, but had some concerns about whether the data as it stood in the ginormous financial database would correlate to the published official records. They made a good point that budget data is always “point in time” and that the organizational chart doesn’t always get reflected in the chart of accounts.

For example, when you reorganize Transit from being in Public Works to its own department or when you take IT out of Finance, all of a sudden your organizational chart doesn’t match up to your financial chart of accounts in the system. It might take weeks or even months until all of the journal entries and modifications to the accounting tables are done. It generally takes a whole year to replace a financial system, since it’s so complex. In order to present a correct financial picture in the budget book, a lot of work is needed to make sure that account X is included in the tally for department Y.

The answer in this case was to provide actual working documents that were used to build the official budget book.

This didn’t solve everything by any means (though the app did get deployed), and there might be people who might argue that the point in time database would be interesting to dig into. This is a good example of how citizens and government staff speak a different language and have different perspectives. They need an advocate who understands and has compassion for both sides.

As with the general population, there are outliers in both government staff and citizens (just like there are the Bernie Madoffs, Mark Sanfords, and Anthony Weiners), but generally, we’re talking about normal people simply trying to do their jobs and make a difference in the world. Government staff isn’t (generally) trying to hide anything. Citizens (generally) aren’t trying to play gotcha games with staff. Government staff is (generally) just prioritizing their work based on what their bosses want and what regulatory requirements and processes they’re following. Citizens are (generally) simply trying to ask a simple question and get a simple answer. An open data advocate who understands both sides of the coin can help create more of a conversation and less of a series of demands and silence.

But the question then becomes, how come processes don’t integrate the need for open data, and how come the bosses don’t think open data is a good idea?

Well, I think that they will one day. President Obama’s Open Data executive order issued in May 2013 is a great start for federal agencies. For cities, I think that there will be increased adoption as leadership begins to understand that there is significant efficiency to be gained by adopting open data practices and processes—as we’ve demonstrated in Asheville. As general practices, like procurement, start to integrate the need for a more efficient operation than mere “open records” into the process, open data will become an important requirement. What if RFPs and RFQs mandated that vendors address “open data” requirements in their proposals from the beginning? Or offered the proposals in a form where it was easy to separate “proprietary by the state law definition” from “public record”?

The bottom line is that good leadership recognizes that wasting everyone’s time on redacting, inspecting, and repackaging documents and data is a bad idea.

Good leadership will recognize the pragmatics of open data.

About the Author

Jonathan Feldman is Chief Information Officer for the City of Asheville, North Carolina, where his business background and work as an InformationWeek columnist have helped him to develop innovation through better business technology and process. Asheville is a rapidly growing and popular city; it has been named a Fodor top travel destination, and is the site of many new breweries, including New Belgium’s east coast expansion. During Jonathan’s leadership, the City has been recognized nationally and internationally (including the International Economic Development Council New Media, NATOA Community Broadband, and the GMIS Best Practices awards) for improving services to citizens and reducing expenses through IT innovation.

References

• Forbes, D. (2012, November 15). Age of Access: Asheville’s budding open-data push. Mountain Xpress. Retrieved from http://www.mountainx.com/article/46806/Age-of-Access-Ashevilles-budding-open-data-push
• Henderson, B. & Steele, C. (2013, March 3). N.C. sheriffs wage fight over gun records. The Charlotte Observer. Retrieved from http://www.charlotteobserver.com/2013/03/03/3889773/nc-sheriffs-wage-fight-over-gun.html#.UiZBV2RATy0
• North Carolina General Statutes, Ch. 132 (2013). Retrieved from http://www.ncleg.net/EnactedLegislation/Statutes/PDF/ByChapter/Chapter_132.pdf