Monthly Archives: June 2014

Predictive analytics in the city

Predictive analytics is a way of responding to and taking advantage of historical and emerging, large datasets by using “a variety of statistical, modeling, data mining, and machine learning techniques … to to make predictions about the future”. One of the advantages of drawing predictions from big data, according to the vision of social physics, is that

You can write down equations that predict what people will do. That’s the huge change. So I have been running the big data conversation … It’s about the fact that you can now understand customers, employees, how we organise, in a quantitative, predictive way for the first time.

Predictive analytics is fervently discussed in the business world, if not fully taken up, and increasingly by public services, governments or medical practices to exploit the value hidden in the public archive or even in social media. In New York for example, there is a geek squad to Mayor’s office, seeking to uncover deep and detailed relationships between the people living there and the government, and at the same time realising “how insanely complicated this city is”. In there, an intriguing question remains as to the effectiveness of predictive analytics, the extent to which it can support and facilitate urban life and the consequences to the cities that are immersed in a deep sea of data, predictions and humans.

Let’s start with an Australian example. The Commonwealth Scientific and Industrial Research Organisation (CSIRO) has partnered with Queensland Health, Griffith University and Queensland University of Technology and developed the Patient Admission Prediction Tool (PAPT) to estimate the presentations of schoolies, leavers of Australian high schools going on week-long holidays after final examines, to nearby hospitals. The PAPT derives their estimates from Queensland Health data on schoolies presentations in previous years, including statistics about the numbers of presentations, parts of the body injured and types of these injuries. Using the data, the PAPT benefits hospitals, their employees and patients by improved scheduling of hospital beds, procedures and staff, with the potential of saving $23 million per annum if implemented in hospitals across Australia. As characterised by Dr James Lind, one of the benefits of adapting predictive analytics is the proactive rather than reactive approaches towards planning and management:

People like working in a system that is proactive rather than reactive. When we are expecting a patient load everyone knows what their jobs [are], and you are more efficient with your time.

The patients are happy too, because they receive and finish treatment quickly:

Can we find such success when predictive analytics is practised in various forms of urban governance? Moving the discussion back to US cities again and using policing as an example. Policing work is shifting from reactive to proactive in many cities, in experimental or implementation stages. PredPol is predictive policing software produced by a startup company and has caught considerable amount of attention from various police departments in the US and other parts of the world. Their success as a business, however, is partly to do with by their “energetic” marketing strategies, contractual obligations of referring the startup company to other law enforcement agencies, and so on.

Above all, claims of success shown by the company are difficult to sustain in closer examination. The subjects of the analytics that the software focuses are very specific: burglaries, robberies, vehicle thefts, thefts from vehicles and gun crimes. In other words, the crimes that have “plenty of data to chew on” for making predictions, and are of the opportunistic crimes which are easier to prevent by the presence of the patrolling police (more details here).

This further brings us to the issue of the “proven” and “continued” aspects of success. These are even more difficult and problematic aspects of policing work for the purpose of evaluating and “effectiveness” and “success” of predictive policing. To prove that an algorithm performs well, expectations for which an algorithm is built and tweaked have to be specified, not only for those who build the algorithm, but also for people who will be entangled in the experiments in intended and unintended ways. In this sense, transparency and objectivity related to predictive policing are important. Without acknowledging, considering and evaluating how both the crimes and everyday life, or both normality and abnormality, are transformed into algorithms and disclosing them for validation and consultation, a system of computational criminal justice can turn into, if not witchhunting, alchemy – let’s put two or more elements into a magical pot, stir them and see what happens! This is further complicated by knowing that there are already existing and inherent inequalities in crime data, such as reporting or sentencing, and the perceived neutrality of algorithms can well justify cognitive biases that are well documented in justice system, biases that could justify the rational that someone should be treated more harshly because the person is already on the black list, without reconsidering how the person gets onto such list in the first place. There is an even darker side of predictive policing when mundane social activities are constantly treated as crime data when using social network analysis to profile and incriminate groups and grouping of individuals. This is also a more dynamic and contested field of play considering that while crime prediction practitioners (coders, private companies, government agencies and so on) appropriate personal data and private social media messages for purposes they are not intended for, criminals (or activists for that matter) play with social media, if not yet prediction results obtained by the reverse engineering of algorithms, to plan coups, protests, attacks, etc.

For those who want to look further into how predictive policing is set up, proven, run and evaluated, there are ways of opening up the black box, at least partially, for critically reflecting upon what exactly it could achieve and how the “success” is managed both in computer simulation and in police practices. The Chief scientist of PredPol gave a lecturer where, as pointed out:

He discusses the mathematics/statistics behind the algorithm and, at one point, invites the audience not to take his word for it’s accuracy because he is employed by PredPol, but to take the equations discussed and plug in crime data (e.g. Chicago’s open source crime data) to see if the model has any accuracy.

The video of the lecturer is here

Furthermore, RAND provides a review of predictive policing methods and practices across many US cities. The report can be found here and analyses the advantages gained by various crime prediction methods as well as their limitations. Predictive policing as shown in the report is far from a crystal ball, and has various levels of complexity to run and implement, mathematically, computationally and organisationally. Predictions can range from crime mapping to predicting crime hotspots when given certain spatiotemporal characteristics of crimes (see a taxonomy in p. 19). As far as prediction are concerned, they are good as long as crimes in the future look similar to the ones in the past – their types, temporality and geographic prevalence, if the data is good, which is a big if!. Also, predictions are good when they are further contextualised. Compared with predicting crimes without any help (not even from the intelligence that agents in the field can gather), applying mathematics to help in a guessing game creates a significant advantage, but the differences among these methods are not as dramatic. Therefore, one of the crucial messages intended by reviewing and contextualising predictive methods is that:

It is important to bear in mind that the predictive methods discussed here do not predict where and when the next crime will be committed. Rather, they predict the relative level of risk that a crime will be associated with a particular time and place. The assumption is always that the past is prologue; predictions are made based on the analysis of past data. If the criminal adapts quickly to police interventions, then only data from the recent past will be useful to police departments. (p. 55)

Therefore, however automated, human and organisational efforts are still required in many areas in practice. Activities such as finding relevant data, preparing them for analysis, tweaking factors, variables and parameters, all require human efforts, collaboration as a team and transforming decisions into actions for reducing crimes at organisational levels. Similarly, human and organisational efforts are again needed when types and patterns of crimes are changing, targeted crimes shift, results are to be interpreted and integrated in relation to changing availabilities of resources.

Furthermore, the report reviews the issues of privacy, transparency, trust and civil liberties within existing legal and policy frameworks. However, it becomes apparent that predictions and predictive analytics need careful and mindful designs, responding to emerging ethical, legal and social issues (ELSI) when the impacts of predictive policing occur at individual and organisational levels, affecting the day-to-day life of residents, communities and frontline officers. While it is important to maintain and revisit existing legal requirements and frameworks, it is also important to respond to emerging information and data practices, and notions of “obscurity by design” and “prodecural data due processes” are ways of rethinking and designing relationships between privacy, data, algorithms and predictions. Even the term transparency needs further reflections to make progress on issues concerning what it means under the context of predictive analytics and how it can be achieved by taking into account renewed theoretical, ethical, practical and legal considerations. Under this context, “transparent predictions” is proposed wherein the importance and potential unintended consequences are outlined with regards to rendering prediction processes interpretable to humans and driven by causations rather than correlations. Critical reflections on such a proposal are useful, for example this two part series – (1)(2), further contextualising transparency both in prediction precesses and case-specific situations.

Additionally, IBM has partnered with New York Fire Department and Lisbon Fire Brigade. The main goal is to use predictive analytics to make smart cities safer by using the predictions to better and more effectively allocate emergency response resources. Similarly, crowd behaviours have already been simulated for understanding and predicting how people would react in various crowd events in places such as major traffic and travel terminals, sports and concert venues, shopping malls and streets, busy traffic areas, etc. Simulation tools take into account data generated by sensors, as well as quantified embodied actions, such as walking speeds, body sizes or disabilities, and it is not difficult to imagine that more data and analysis could take advantage of social media data where sentiments and psychological aspects are expected to refine simulation results (a review of simulation tools).

To bring the discussion to a pause, data, algorithms and predictions are quickly turning not only cities but also many other places into testbeds, as long as there are sensors (human and nonhuman) and the Internet. Data will become available and different kinds of tests can then be run to verify ideas and hypotheses. As many articles have pointed out, data and algorithms are flawed, revealling and reinforcing unequal parts and aspects of cities and city lives. Tests and experiments, such as manipulating user emtions by Facebook in their experiments, can make cities vulnerable too, when they are run without regards to embodied and emotionally chared humans. Therefore, there is a great deal more to say about data, algorithms and experiments, because the production of data and experiments of making use of them are always an intervention rather than evaluation. We will be coming back to these topics in subsequent posts.

Post in 'Big Data, Big Questions' series on LSE Impact blog

Rob Kitchin provides the first interview in a new series on the LSE Impact blog entitled ‘Big Data, Big Questions’ curated by Mark Carrigan.  The post concerns the philosophy of data science, the nature of ‘big data’,  the opportunities and challenges presented for scholarship with its growing influence, the hype and hubris surrounding its advent, and the distinction between data-driven science and empiricism.  You can read more over at the LSE Impact blog.

Programmable City researcher nominated as 100 top women in STEM

We are delighted to report that, as part of the Silicon Republic’s Women Invent Tomorrow campaign, Dr Tracey P. Lauriault is one of the 100 top women in the areas of science, technology, engineering and maths (STEM).

In Ireland she has been actively engaged in critical data research on topics related to Open Data, Big Data, Open Government and Infrastructures.  Like other members of the Programmable City Team she is doing field work in Dublin and Boston.  Because her home country is Canada, that has been added.  She is also a big supporter of evidence based decision making and deliberative democracy, and considers numbers, data and open and interoperable infrastructures as being part of that process.

Congratulations to Tracey! And you can read more about other inspiring women over at  Silicon Republic.

Hailo, Uber and the deregulation of Ireland's taxi industry

This afternoon’s black cab blockade of London comes in response to car ride apps that are changing the character of the city’s taxi industry. While there has been little visible backlash against comparable services in Ireland, it seems only a matter of time before these struggles find their way to our shores.

The market-leading smart phone taxi service in Ireland is Hailo. Self-described as “the evolution of the hail”, Hailo was founded in November 2011 by three London taxi drivers and three entrepreneurs. It launched in Dublin, its second city, in July 2012 and as of mid-2014, provides sporadic coverage across the country with a specific attention on areas of high population density.

HailO dominates in Dublin by Brandon Zeman

To use Hailo as a potential passenger, one needs simply to download and launch their free smart phone application. To use the Hailo as a driver, things are a little more complicated. One must sign up using a geographically specific online portal (such as exists for London, Ireland, New York, Boston and Tokyo). Registration is restricted to licensed taxi drivers such that Hailo is effectively leveraging the screening procedures of existing small public service vehicle (SPSV) infrastructure in Ireland. Of Dublin’s 12,000 registered taxi drivers, I’ve been told that about half are using the service.

In this post I will describe two observations on the role Hailo plays in Dublin: that it competes with existing taxi infrastructure, and that it capitalises on and potentially extends the deregulation of transportation in the city. I will briefly compare the service to Uber and Lyft, and argue why their competition will likely bring the taxi wars to Dublin.

Hailo competes with existing taxi infrastructure

In addition to plying for hire (being in motion and available for hire) and standing for hire (being stationary and available for hire), taxi drivers can increase their number of fares by enrolling to a radio service. When a customer phones a taxi company, this company then leverages its radio-enabled network to source an available taxi driver. Cab drivers working in Dublin have told me that subscription to such a service can cost as much as €5,000 per year. This service is dependent upon a radio communications unit being physically installed into driver’s vehicle, the hire of which is presumably incorporated into the cost of the service.

Hailo, in drawing upon already existing smart phone usage, does not need to install any radio communications infrastructure. This means lower fixed capital costs and lower associated installation and maintenance costs. Furthermore, by having software perform the role of the radio operator, there are presumably fewer attendant labour costs.

These factors lead to a considerably different pricing model. Rather than charge a yearly subscription fee, Hailo is free to install and use, but charges a 12% commission on every fare sourced through the application. In order to compete with these rates, a €5,000 per year radio service would need to direct €41,667 worth of fares to each driver. Understandably, Hailo poses a considerable threat to Dublin’s taxi radio companies.

There is however an important geographical caveat which needs to be made. In cities such as Dublin – where there is a confluence of high population density, a high number of taxis per head and a high usage of smart phones – the benefits of Hailo to both drivers and passengers outweigh those offered by taxi radio companies. In a geographical location where taxi or smart phone use is more sparsely distributed however, Hailo has less opportunity to draw upon existing infrastructure. Where I work in Maynooth – a small university town 20 kilometres west of Dublin – it is very difficult to find a taxi using Hailo. In such locations both spatial scarcity and community loyalty lead me to suppose far less competition between Hailo and existing taxi radio services. In these instances, the volume of jobs rather than rate of commission is the important factor.

Hailo availability in Dublin versus Maynooth

Competing SPSV smart phone service Uber, which has been recently valued at a truly astounding $18.2bn1, launched in Dublin in January 2014. Uber operates under a business model which is far more challenging to existing taxi infrastructures as a whole. Rather than recruit taxi drivers exclusively, Uber is open to private hire vehicle (or limousine) drivers as well. Less regulation on the cost of limousine services allows the company to employ a surge pricing model, so that fares can cost considerably more than a standard rate (up to as much as eight times more expensive in rare incidences of peak demand). The cost of an SPSV licence for limousines in Ireland is only €250, compared to the €6,300 for a taxi licence. While cars must still be deemed as fit for such a purpose, and drivers must similarly undergo clearance by An Garda Síochána, Uber hopes that its matchmaking infrastructure for limousine services will allow its service to compete favourably with the taxi industry as a whole. It’s probably too soon to draw any conclusions on the success of the company in Dublin, but you can be sure they are here for the long run. Uber have money to burn and their CEO Travis Kalanick has a combative attitude toward vested interests.

Hailo capitalises on and potentially extends the deregulation of transportation in the city

As described above Hailo is most effective in urban areas where there is already considerable competition amongst taxi drivers. Uptake amongst drivers is dependent upon a pull effect, whereby not using the service would render them excluded from a proportion of the passenger market. There is, as such, a supply-and-demand-like positive feedback loop between the driver and passenger applications. Increased use of the application by one group would be expected to result in an increase in use by the other.

It was not always so easy to hail a taxi in Dublin. Between 1978 and 2000 local authorities in Ireland were entitled to limit the number of taxi licences issued in their area. In Dublin, the number of licences in circulation between 1978 and 1988 was fixed at 1,800. This number was increased slowly through the late 1980s and 1990s to around 2,800 by the end of the decade. Supply was not kept up with demand however. By the late 1990s the cost of purchasing a licence on the open market was as much as €100,000. In line with its commitment to improving Dublin’s taxi services, the Action Programme for the Millennium encouraged the issuance of 3,100 new licences in the city in November 1999. This was not found to be enough however, as one year later, on November 21, 2000, S.I No. 367/20002 lifted licensing regulations in Ireland. This had the immediate effect of devaluing existing licences and subsequent enquiries and legal cases have been undertaken to assess the fairness of this deregulation. The change has had its intended effect however. By 2008, union estimates placed the number of taxi licences nationwide at 19,000 with 12,000 of those being held in Dublin. Locals tell me that these days it is much easier to get a cab home after a night out than it was in the 1990s.

Under conditions where taxi drivers do not have to compete so vigorously for fares (under a licence-restricted, consortium-based or heavily unionised model rather than the predominantly individualised system currently in operation in Ireland), there would be little pressure for them to use Hailo. And as I’ve already argued, no drivers using the service would translate to no passengers using it either. Indeed it is quite clear from various interviews and media appearances that Hailo have made tactical decisions about how and in which cities they introduce their service.

While Hailo takes advantage of such spaces of deregulation, it has been careful so far to adhere to local legislation. The service does not on its own necessitate a further deregulation of the SPSV sector in Ireland. Hailo is not unique however. Competitors include the globe-spanning Uber, Lyft, Sidecar, Wingz, Summon and Flywheel in the USA, cab:app, Cab4Now, Get Taxi and Kabbee in London and Chauffeur-Privé in France. These transportation network companies compete on a range of business models, services and geographical coverage. This marketplace of competing car ride apps has the potential to push against regulations which are in place to control regional SPSV sectors in Ireland.

Consider the following examples from what’s been called the taxi wars.

Since March 2014, Republican Senator Marco Rubio has been pushing for the deregulation of the limousine industry in Miami, Florida specifically to make way for the introduction of Uber. The taxi and limousine industry in Miami-Dade county is strongly regulated. The industry caters to a population of over 2,500,000 people through 2,121 taxi medallions (which is the equivalence of a vehicle licence in Ireland). This has forced the market price of a medallion to around $340,000, 28% of which are owned by taxi drivers. Similarly, limousines licences are limited to 625 in number. Services must be booked an hour in advance with rides costing a minimum of $70. This regulation is directly restrictive of Uber’s business model. After Senator Rubio’s efforts to clear the company’s way stalled, actor Ashton Kutcher was recruited to tackle regulation from a different front, decrying the city’s bizarre, old, antiquated legislation” on Jimmy Kimmel Live. On May 22 rival company Lyft – “your friend with a car”, which adopts a decentralised ride sharing model – launched in Miami on a donation-based payment scheme. This forced a response by Uber, which launched its comparably modelled UberX service in the city on June 4. These are not isolated events. Similar changes are being forced on other regions of the US such as: Arlington, Virginia; San Antonio, California; and Austin, Texas.

London Cabs by rosipaw

Uber arrived in London, it’s first European city, in mid-2012. London’s taxi industry consists of over 25,000 hackney carriage taxis serving a daytime population of around 10,000,000. In order to obtain a taxi licence, or Green Badge, drivers must pass to a knowledge test covering a 113 square mile area of the city which is purported to require up to five years of study. Uber, in accepting any driver with a private hire vehicle licence, poses a clear threat to the time and money that London’s taxi drivers have invested in their profession. In recent weeks, The Licensed Taxi Drivers Association has summoned six Uber drivers to court, alleging that the Uber smartphone app is equivalent to a taximeter and therefore illegal under the 1998 Private Hire Act which reserves that right for licensed taxi drivers. In addition, the London taxi-driver’s union lambast Transport for London for failing to pursue the illegal activities which Uber facilitate, suggesting that the state body is afraid of the money backing Uber. In response, Transport for London has sought a binding decision from the high court regarding the legality of the smart phone taximeter.

The way in which other transportation network companies have responded to Uber’s entrance into the London market is important in the case of Ireland. Hailo have announced that they will extend their services to private hire vehicles in addition to taxis. This was news was not well accepted by the industry, with the word ‘scabs’ being gratified on the wall of Hailo’s London office. The company responded on their website, insisting that they are simply following consumer demand. Assuming the move doesn’t too badly damage its position among existing service users, we can expect to see something similar in Ireland, either to pre-empt or combat Uber’s growing market share. Not all apps are going the way of private vehicle hire however. Competitor Cab:app has pushed back against the trend set by Uber and Hailo, asserting publicly its commitment to the taxi industry.

Protests in London and throughout Europe today (June 11, 2014) are in direct opposition to Uber’s perceived infringement upon the taxi industry. In the past, strike action has escalated to involve the destruction of property. In January 2014, Parisian taxi drivers struck in opposition to Uber’s unregulated market competition. Confrontations between unionised taxi drivers and Uber drivers resulted in “Smashed windows, tires, vandalized vehicle[s], and bleeding hands”.

The Irish SPSV sector is admittedly quite different from comparable industries in America, the UK and France. Hailo does not face the same kind of regulatory barriers in Ireland as it would in parts of the US. The National Taxi Drivers’ Union is less powerful than taxi unions in London, and certainly less militant than those in Paris. Hailo is a relatively well behaved company in this sector however. As competitors such as Uber and Lyft seek to expand their more aggressive strategies throughout Europe, it is highly likely, given the success of Hailo and Dublin’s reputation as a high-tech-friendly city, that a real push will be made to establish a foothold in Ireland. Spokesperson for Uber’s international operations Anthony El-Khoury has told the Irish Times: “We see a lot of potential in the Irish market and a lot of demand”. As this market heats up, pressure to further deregulate the SPSV industry will be put on Dublin City Council and the Irish government.

The future of the SPSV sector in Ireland

These two preliminary observations on the role of Hailo in Dublin are commensurate with larger tendencies of state deregulation and market-oriented competition. While the emergence of car ride apps may be positioned as a form of Schumpeterian creative-destruction, there are larger political and economic forces within which these applications should be contextualised. I have attempted to sketch some of these forces in this blog post.

The taxi industry is regulated for a reason. The screening and approval of drivers is important in ensuring accountability and the safety of passengers. Regular metering and Ireland’s nationwide fare system (which was instituted in September 2006) make certain that taxis provide an honest and consistent service that does not gouge rural communities or those in need. While Hailo operates under a minimally challenging model to these standards, it does not exist in a vacuum. By exploiting the freer regulation of limousines, Uber is effectively side-stepping many but not all of them. By doing away with accreditation altogether, Lyft would throw these standards onto the wills of the market. Car ride apps have the potential to put huge pressure on the taxi and limousine business. If other European cities are anything to go by Ireland’s SPSV sector is likely to be forced to deregulate further in coming years. This impacts the price, availability and safety of taxi services, and undercuts the ability of workers in the industry to bargain for fair pay and work conditions. It is important that any changes to the sector be submitted to proper public scrutiny and debate.

Jim Merricks White

Updates

June 12 – An article on RTE alerted me to the recent launch of Wundercar in Dublin. The app seems to follow the Lyft or UberX business model, whereby unlicensed drivers give passengers a lift somewhere in the city for “tips”. Drivers are screened by Wundercar rather than An Garda Síochána.

June 22 – The Independent reported on June 20 that Hailo is offering a suite of new services targeting the business sector. Most significant to my discussion here is the introduction of the company’s limousine option to Ireland. Limousine licences are considerably cheaper than taxi licences and there are fewer regulations on fare pricing.


  1. Aswath Damodaran of New York University claims that a more accurate valuation would be around $6bn. By way of comparison Hailo was valued, 18 months ago, at a far more modest $140m.

  2. Also called Road Traffic (Public Service Vehicles) (Amendment) (No. 3) Regulations 2000.

Mapping Openness and Transparency

by: Tracey P. Lauriault

I attended the European Regional Meeting of the Open Government Partnership at the Dublin Castle Conference Centre in May of this year.  The meeting was a place for performance and evaluation wonks to show their wares, especially at the following sessions: Open Government Standards and Indicators for Measuring Progress, The EU’s Role in Promoting Transparency and Accountability and Engagement with the OGP, and Open Contracting: Towards a New Global Norm.  I did not attend the Independent Reporting Mechanism (IRM) sessions, but having read the IRM report for Canada, I know that it too is an emerging performance evaluation indicator space, which is affirmed by a cursory examination of the IRMs two major databases.  The most promising, yet the most disappointing session was the Economic Impact of Open Data session.  This is unfortunate as there are now a number of models by which the values of sharing, disseminating and curating data have been measured.  It would have been great to have heard either a critical analysis or a review of the newly released Ordinance Survey of Ireland report, Assessment of the Economic Value of the Geospatial Information Industry in Ireland, the many economic impact models listed here in the World Bank Toolkit, or the often cited McKinsey Global Institute Open data: Unlocking innovation and performance with liquid information report.  Oh Well!

While there I was struck by the number of times maps were displayed.  The mapping of public policy issues related to openness seems to have become a normalized communication method to show how countries fare according to a number of indicators that aim to measure how transparent, prone to corruption, engagemed civil society is, or how open in terms of data, open in terms of information, and open in terms of government nation states are.

What the maps show is how jurisdictionally bound up policy, law and regulatory matters concerning data are.  The maps reveal how techno-political processes are sociospatial practices and how these sociospatial matters are delineated by territorial boundaries.  What is less obvious, are the narratives about how the particularities of the spatial relations within these territories shape how the same policies, laws and regulation are differentially enacted.

Below are 10 world maps which depict a wide range of indicators and sub-indicators, indices, scorecards, and standards.  Some simply show if a country is a member of an institution or is a signatory to an international agreement.  Most are interactive except for one, they all provide links to reports and methodologies, some more extensive than others.  Some of the maps are a call to action; others are created to solicit input from the crowd, while most are created to demonstrate how countries fare against each other according to their schemes.  One map is a discovery map to a large number of indicators found in an indicator portal while another shows the breadth of civil society participation.  These maps are created in a variety of customized systems while three rely on third party platforms such as Google Maps or Open Street Maps.  They are published by a variety of organizations such as transnational institutions, well resourced think tanks or civil society organizations.

We do not know the impact these maps have on the minds of the decision makers for whom they are aimed, but I do know that these are often shown as backdrops to discussions at international meetings such as the OGP to make a point about who is and is not in an open and transparent club.  They are therefore political tools, used to do discursive work.  They do not simply represent the open data landscape, but actively help (re)produce it.  As such, they demand further scrutiny as to the data assemblage surrounding them (amalgams of systems of thought, forms of knowledge, finance, political economies, governmentalities and legalities, materialities and infrastructures, practices, organisations and institutions, subjectivities and communities, places, and marketplaces), the instrumental rationality underpinning them, and the power/knowledge exercised through them.

This is work that we are presently conducting on the Programmable City project, which will  complement a critical study concerning city data, indicators, benchmarking and dashboards, and we’ll return to them in future blog posts.

1.       The Transparency International Corruption by Country / Territory Map

Users land on a blank blue world map of countries delineated by a thick white line, from which they select a country of interest.  Once selected a series of indicators and indices such as the ‘Corruption measurement tools’, ‘Measuring transparency’ and ‘Other governance and development indicators’ appear.  These are measured according rankings to a given n, scored as a percentage and whether or not the country is a signatory to a convention and if it is enforced.  The numbers are derived from national statistics and surveys.  The indicators are:

  • Corruption Perceptions Index (2013), Transparency International
  • Control of Corruption (2010), World Bank dimension of Worldwide Governance Indicators
  • The Bribe Payer’s Index (2011), Transparency International
  • Global Corruption Barometer (2013), Transparency International
  • OECD Anti-Bribery Convention (2011)
  • Financial Secrecy Index (2011), Tax Justice Network
  • Open Budget Index (2010), International Budget Partnership
  • Global Competitiveness Index (2012-2013), World Economic Forum Global Competitiveness Index
  • Judicial Independence (2011-2012), World Economic Forum Global Competitiveness Index
  • Human Development Index (2011), United Nations
  • Rule of Law (2010), World Bank dimension of Worldwide Governance Indicators
  • Press Freedom Index (2011-2012) Reporters Without Borders
  • Voice & Accountability (2010), World Bank dimension of Worldwide Governance Indicators

By clicking on the question mark beside the indicators, a pop up window with some basic metadata appears. The window describes what is being measured and points to its source.

The page includes links to related reports, and a comments section where numerous and colourful opinions are provided!

2.      Open Government Standards

Users land on a Google Map API mashup of Government, Citizen and Private Open Government initiatives.  They are given the option to zoom in to see local initiatives.  In this case, users are led to a typology of initiatives which define what Open Government means from civil society’s point of view.

Initiatives are classified with respect to the following categories 1) Transparency, 2) Participation and 3) Accountability.  The development of the Open Government Standards are being coordinated by “Access Info Europe, a human rights organisation dedicated to the promotion and protection of the right of access to information in Europe and the defence of civil liberties and human rights with the aim of facilitating public participation in the decision-making process and demanding responsibility from governments”.

Definitions, parameters and criteria for a number of sub-indicators are being crowsourced in the online forms like the following for Openness:

The following is a list of standards that are a currently under development.

  • Recognition of the Right to Know
  • Openness
  • Codes of Conduct: Clear standards of behaviour
  • All information available from all public bodies
  • Clear and reasonable Timelines
  • Conflict of Interest Prevention Mechanisms
  • Access is the Rule – Secrecy is the Exception
  • Clear and comprehensive information
  • Assets Disclosure
  • Proactive Publication of Information
  • Active collaboration
  • Transparency and Regulation of Lobbying
  • Free of charge and free for reuse
  • Appropriate and Clear Procedures
  • Whistleblower mechanisms and protections
  • Open Formats
  • Empowerment
  • Procurement Transparency
  • Compilation of information
  • Transparency and Accountability
  • Independent Enforcement Bodies
  • Independent review mechanism

3.      The Global Integrity Report Map

This is an interactive Open Street Map (OSM) Mapbox map depicting the locations where there is Global Integrity fieldwork national reports arranged by the year these were published.  Reports are called Country Assessments and each includes: a qualitative Reporter’s Notebook and a quantitative Integrity Indicators scorecard.

The Integrity Indicators scorecard assesses “the existence, effectiveness, and citizen access to key governance and anti-corruption mechanisms through more than 300 actionable indicators. They are scored by a lead in-country researcher and blindly reviewed by a panel of peer reviewers, a mix of other in-country experts as well as outside experts. Reporter’s Notebooks are reported and written by in-country journalists and blindly reviewed by the same peer review panel”.

Users select a country, and below the map a number of scorecard indicators appear.  Scorecard indicators are arranged into 6 major categories:

  1. Non-Governmental Organizations, Public Information and Media
  2. Elections
  3. Government Conflicts of Interest Safeguards & Checks and Balances
  4. Public Administration and Professionalism
  5. Government Oversight and Controls
  6. Anti-Corruption Legal Framework, Judicial Impartiality, and Law Enforcement Professionalism

Users can then access how each score was derived by following sub-category links.  Below is an example of legislation and the score associated with the Political Financing Transparency indicator which is a sub-class of the Elections category.

PDF copies of the reports are also available, as are spreadsheets of the data used to derive them.

4.      The World Bank Global Integrity Index Map

This is an interactive map depicting the World Bank’s Global Integrity Index, which is one of its Actionable Governance Indicators (AGIs).  AGIs “focus on specific and narrowly-defined aspects of governance, rather than broad dimensions. These indicators are clearly defined, providing information on the discrete elements of governance reforms, often capturing data on the “missing middle” in the outcome chain”.  The map allows users to select from a drop down menu which includes a subset of AGI indicator – the portal contains thousands.  The interactive and downloadable map aims to graphically demonstrate the progress of governance reform worldwide.  The map is but a small picture of what the Portal contains and below is a Governance At A Glance country report for Ireland.

And here is a data availability table, also for Ireland.

5.      The Open Government Partnership Participating Countries map

The interactive map depicts the countries that have signed onto the Open Government Partnership and in which cohort they belong.  Users can select their country of choice which hyperlinks to that country’s membership status and its progress to date in meeting the criteria for membership, where it ranks in terms of commitment and links to related documents such as action plans and progress reports.  It is interesting to note, that the Independent Review Mechanism reports are not included in this list.  Canada’s IRM report was submitted in 2014.

6.      The Open Data Barometer Data Map

The Open Data Barometer map depicts the 77 countries the Open Data Institute has evaluated.  This map assesses how open data policies are implemented in these countries according to three main indicators:

  • Readiness of:
    • government,
    • citizens and civil society and
    • entrepreneurs and business,
    • Implementation based on the availability a variety of datasets within the following sub-categories:
      • accountability
      • social policy
      • Innovation
    • The following Emerging Impacts :
      • Political
      • Social
      • Economic

These are also graphically depicted in a radar chart, as well as a bubble chart where the size of the bubble represents the availability of the datasets per category and if these sets meet the Open Definition Criteria.  The data and associated methodologies are explained in the website about the report.

7.      Open Contracting Implementation and Supporting Tools Map

This is a static map depicting where Open Contracting Support and Tools are implemented.  Sadly, the 5 indicators depicted on the map were not explained or described.  I would have to contact them at a later time to find out!

8.      Reporters Without Borders World Press Freedom Index Map

This map is depicting the findings of the World Freedom Index Report for 2014, with a particular focus on how countries rose and fell from the previous year.  180 Countries are scored against the following criteria which are based partly on a questionnaire, violence committed against journalists the algorithm of which is clearly defined in the PDF copy of the report.  Data and the report are fully downloadable, and more detailed maps with a legend are provided in the report itself and a methodology report.

  • Pluralism
  • Media independance
  • Environment and self-censorship
  • Legislative framework
  • Transparency
  • Infrastructure

9.      Politics for People Not Profit Map

This is an interactive map that depicts the pledges made by nationally elected political officials to the European Parliament and asking them to commit to “stand-up for citizens and democracy against the excessive lobbying influence of banks and big business in the EU?” Mousing over a country triggers a pop-up menu which lists which party has made a commitment, while clicking on the map directs users to the page below which is a tool whereby citizens can fill in a form letter and have it sent to their elected officials to solicit them to pledge.

10.      The OGP Civil Society Hub Map

This is a partially curated and partially crowsourced map on the OGP Civil Society Hub Website.  The starred drops represent countries that are official OGP members, while the red drops represent any number of civil society organizations that have in some way engaged with the OGP, some of which are transnational while others are national or sub-national entities.  Once a location is selected, a pop-up menu appears that includes a national flag, a link to that nation state’s official OGP member site, and provides users with the option to pick from who is involved, a selection of topic areas or a list of information resources.  The map is a means to find people and activities and also a means by which to have people self identify and be recognized as civil society actors but also to connect people.

Unfortunately, the very popular and often discussed Open Knowledge Foundation Open Data Index, the Open Corporate Data indices and the Open Data Study by the Open Society Foundation have not been mapped, even though the latter includes quite a lovely world map on the cover of its report.