We had a lot of fun putting together this interactive on singles, conception dates, and sexually transmitted infections. There are a lot of nice maps and interactive graphics, all wrapped up in a Tableau package.
Aside from being a fun feature for Valentine’s Day, it was also an experiment in using Tableau as a navigation tool for a complex, multifaceted feature.
It was great to find out that you could embed Fusion Tables maps into Tableau. It’s also cool that you can use Tableau to essentially set up a slideshow between elements.
It’s not the neatest tool for navigation though, and it’s really unnecessarily complex to create buttons that let you move between slides. In the end, it was a lot of work.
I’d try it again, but probably for something a little simpler and with a little more time to prepare.
And yes, we purposely released STI maps on Valentine’s Day - or VD, as some people on my Facebook feed are calling the holiday.
Interactive: Increasingly, Ontario graduates shun teacher's college
A story/interactive package I’ve been working on for a while went live yesterday - a look at the very depressing job situation for freshly trained Ontario teachers, and how the production of new teachers got badly out of whack with teacher retirements (teacher education was expanded in response to what turned out to be a temporary surge in retirements in the late ’90s, and is just now being scaled back).
In turn, graduates of Ontario universities are showing much less interest in going to teacher’s college. Under the circumstances, it’s hard to blame them.
I find myself often referencing income maps, in one way or another, when mapping other things (Toronto’s male-victim homicide map, for example, tracks the city’s low-income neighbourhoods fairly precisely).
So it’s helpful, once in a while, to publish the income maps themselves as a reference point, which brings us to this week’s #graphicmonday.
Since Fusion Tables can display FSA-based maps covering the whole country, it’s now possible to publish a hyperlocal map like this on a national scale, in this case a one-stop-shop map of median family income by postal area. The story also has a list of Canada’s ten poorest and ten richest postal code, in some ways surprising, in some ways not.
No, we weren’t late posting this week’s Graphic Monday. But, we were a little late posting it to the Tumblr. Sorry!
I hope you’ll enjoy it though. This week, we used some new Statistics Canada data to put together an interactive graphic of the health problems of different Canadian cities. Sudbury and Peterborough, for example, seem to have serious problems with diabetes and high blood pressure.
My colleague Leslie Young for most of January has been putting out our #graphicmonday feature, which is a way of getting value out of a mass of material we have around that’s interesting, but doesn’t fit a conventional story format. (Although it has turned out that some of them did become stories, on second thought.)
The 2006 data is the most recent available. In 2011, Statistics Canada asked a question designed to count same-sex households, but the question was poorly designed and didn’t distinguish clearly between same-sex couples and two people of the same sex sharing accomodation. At the last minute (actually the morning of the census release), StatsCan decided not to release the data. My efforts to pry it loose with an access-to-information request were unsuccessful, in the end.
The main surprise in the data was a sharp difference between Quebec and the rest of Canada: gay and lesbian couples segregate much more from each other in Montreal, Hull and Quebec City than in cities elsewhere in the country.
It’s generally accepted among journalists that in Canada, it’s very hard to get an interesting news story out of the data that governments make available for download on their various “Open Data” sites.
Quite simply, it’s usually very dry material.
Locations of public drinking fountains and city trees might be nice for app developers, but rarely makes for a story that has any impact or tell us something we’d like to know.
So it’s always refreshing to see a fun data set on an open data government site. I recently came across sales data for B.C. Liquor Stores on the Data BC site. Even better, it was broken down by region and drink type. Immediately, I thought that this would make a fun map that people might like to play with.
I downloaded the data set and went looking for population data and geography.
Luckily, this was B.C., so the excellent B.C. Stats site became my source. In my experience, B.C. has in this website the best and easiest-to-navigate source for basic demographic and geographic data in the country. It is much, much easier to find regional populations and boundary shapefiles for B.C. than for Ontario, for example.
This is the second map I’ve done using B.C.’s open data. (The first is here.) I really hope other governments take note of the work Data BC is doing, and that Data BC continues to add interesting datasets to its collection.
This week’s Graphic Monday feature is an interactive look at some of Canada’s arms exports.
I was surprised to see how many Canadian guns went to Denmark, of all places. Much less surprising is that the U.S. is the biggest recipient of Canadian weapons (at least the ones selected).
This piece uses Statistics Canada data, generously provided and formatted for me by the agency. I tend to have very good experiences with them, as they generally send me a very tailored report based on my usually pretty general question.
This week’s question was, “Where do Canadian guns go?” It was inspired by this CP story about how Canadian merchants can now apply to sell fully-automatic rifles to Columbia.
In today’s Graphic Monday, we take a look at bankruptcy data from across Canada. It seems that the Prairies have a much lower rate of bankruptcy than other parts of Canada, which is interesting.
I would have liked to have it all in one big, nationwide map, but unfortunately Canada is a little too wide from east to west for our website and Fusion Tables to easily accommodate. According to some conversations I’ve had on Twitter, this is a common problem for those who do mapping online.
Click the image below to see the feature on GlobalNews.ca.
We’ve decided to start a new weekly feature on GlobalNews.ca : Graphic Mondays.
Every Monday, we will add a new graphic or interactive map (or something we haven’t even thought of yet!) on a different topic.
There’s lots of data out there, and not all of it makes it into a full news investigation. But it’s interesting stuff, and we think this is a great way to make use of some of the bits of information that come across our desks.
This week, we’re taking a close look at Order of Canada appointees and where they’re from. Look forward to lots more fun features to come!
Click below to visit the feature on GlobalNews.ca.
It started with an idea, of course. After hearing so many reports of concrete falling from the Gardiner during the summer of 2012, I wondered if maybe there was something we weren’t hearing. Was there more concrete falling than we knew about? Were there other problems?
This sort of thing tends to be well-documented by cities, so Freedom of Information seemed like the answer.
I made two Freedom of Information requests to the City of Toronto, one for all emails and communications products dealing with falling concrete on the Gardiner, and the other for all engineering and inspection reports.
When I got the information, it was a little more than 2000 pages long. So, I of course had to read it. It was at this stage that words like “punch-through” really jumped out at me.
The feeling was that if we were going to release a story like this online, it would be a shame not to visualize it. The obvious choice for a visualization seemed to be a map, since people were going to want to know where the problems were.
So we wanted to make a map.
The first step was to catalogue each event. This involved deciding on our criteria (loose concrete that presented some kind of risk. Either it had a high chance of falling, or it was above a high traffic area). I took my cues on the criteria from the documents themselves. Many of them are specifically categorized in this way.
This involved putting every document we had into Document Cloud, a tool for sharing and annotating documents, for easy categorization.
So, I read through every report that fit those criteria, looking for any mention of a location. I then entered those details into a spreadsheet and created a Document Cloud reference for every incident.
Almost all of the locations were referred to by “bent number” instead of an intersection. This meant that you would see a reference to “Bent 23-25” for example. So, changing this into a useable latitude/longitude coordinate for mapping purposes required some extra work.
We purchased a technical drawing of the Gardiner that listed bent locations from the City of Toronto. Then I went through each incident, first placing it on a bent, then manually assigning coordinates.
Since we published the story, the Gardiner has been a hot topic at city hall, with city officials responding with press conferences and councillors debating the merits of different plans for the Gardiner. The problems we raised were news to both the public and to councillors, so the debate might have been very different without it.
Hurricane Sandy motivated us to move a project that’s been lurking around for over a year to the front burner: mapping areas that would be at risk of flooding in a similar extreme storm in the GTA. The most obvious starting point is Hurricane Hazel, still Canada’s most lethal natural disaster and the basis of Ontario flood planning ever since. Our story/map package looked at the 42 areas thought to be at risk of destructive flooding in a similar extreme storm.
One typical problem area is the historic village cores of now-suburban, once-agricultural towns in the 905. Many were built around mills, which of course needed access to river water. The resulting communities, however picturesque, would never have been built today where they are.
There were two stories on Global Toronto last night that came from a digital-to-TV news flow.
The first was a look at the pedestrian accident spike that follows the fall time change- 10 years of Toronto collision data shows that the week after the clocks fall back shows a higher accident rate than the weeks on either side. When the time change dates moved starting in 2007, the effect also moved, showing that it isn’t something coincidental having to do with changing light patterns at that time of year. It seems to have to do with sunset suddenly changing its relationship to the evening rush hour.
About five years ago, when I was working at the Star, someone in the graphics department had the idea of mapping the GTA’s census tracts by the top language other than English, based on the 2006 census. It made a fine double-page map feature in print, but the only way we could think of to present it on the Web was by uploading the page proof, which was a .pdf weighing in at 25MB or so. There has to be a better way, I thought, which started me on this whole map business. (Whether there was a better way was debatable, since it would be another couple of years before we figured out how to handle large numbers of polygons more or less painlessly, or get access to them by some method other than hand-drawing them in Google Earth.)
So it’s nice to be able to present an interactive Google Maps-based map of the GTA’s census tracts by the top language other than English, based on the 2011 census (I may go back and do prior years - we have the data). We have Toronto, as well as 14 other Canadian cities.
Vancouver and Montrealhave interesting maps. The maps themselves cover all tracted areas of the country, so populated areas from large towns up are shown.
There’s a new addition to the federal government’s data.gc.ca portal: vehicle recalls from Transport Canada.
This is a dataset I had filed an Access to Information request for a while ago (see the interactive app I built with it here) and after the story was published, I had contacted the Treasury Board’s data.gc.ca people to let them know that this data should be publicly available for download. They replied, saying that they agreed.
And now it’s up. There are two sets: one for all recalls, and one for those within the last sixty days. The former is updated monthly, the latter daily.
We had already uploaded the dataset to our own Open Data page, but it’s good to have updated versions coming out.
It’s certainly useful stuff and just the kind of thing that belongs on an open data site.
Jacques Poitras wondered, looking at our maps of US citizens in Canada , (see previous post) what a map of Canadians in the United States would look like. I started to wonder, too, and (with some help from the US Census media people) found a table of Canadian-born respondents to the US census by county.
Plugged into a county population table, plugged into a county KML file, we have a more or less equivalent map of Canadian-born US residents, who may or may not also be US citizens. From there it was tempting to create a one-stop shop map showing both sides of the border, with density of Americans in Canada in blue and density of Canadians in the US in green.
Within regions, the cross-border flow of people is sometimes reciprocal, sometimes not. The Maine-New Brunswick border region that Jacques wrote about in his bookhas lots of people living on the opposite side of the border from the one they were born on - Aroostook County, Me. is the top American county for Canadian-born residents - but the US counties facing Quebec don’t show the same pattern. (Quebec has relatively few Americans.)
The Niagara and Windsor border regions show the opposite pattern, with proportionately far more Americans on the Canadian side:
The Lake of the Woods/International Falls area is more reciprocal:
… as is the Vancouver/Bellingham region.
A fully interactive version of the map can be found here.
Iraq war, 2004 Bush victory doubled U.S.-source immigration
by Patrick Cain
I have a story up centred on a Tableau graphic and a series of maps looking at U.S. immigration to Canada over the last 50 years or so. The main driver has been armed conflict, most recently the American invasion of Iraq, which along with George W. Bush’s re-election in 2004 seems to have doubled the flow of Americans here over the course of a few years. (It ended abruptly when Obama was elected, and has been falling.)
The Iraq-related spike was the largest since the Vietnam War, and (deducting the existing pre-2003 levels of about 5,000 U.S. immigrants a year) seems to have led about 20,000 more Americans to immigrate here than would otherwise have been the case. Here’s a screenshot of the graphic (click to see the real thing):
The 2006 census yielded the basis for mapping U.S. citizens by postal code. Map geeks (like me) will want to spend some time with them. Some patterns, like the concentrations along the Maine-New Brunswick border, weren’t a surprise, but I was startled by the large numbers of Americans all the way through the B.C. interior and the Yukon. (The Yukon is the top province/territory for U.S. citizens.)
(Also, many years after the Vietnam War made it a mecca for draft evaders, the Annex is still Ontario’s top postal code for U.S. citizens.)
All the single ladies (and men) - fun with census data
By Leslie Young
As an addition to our census coverage, I decided to take a more lighthearted look at our Statistics Canada spreadsheets, and turn them into a Census Dating Guide.
It was a lot of fun to make. It involved pulling out the unmarried, not common law individuals, male and female, from all age groups and analyzing the data.
We found that St. John’s and (oddly) Saint John had the highest ratio of available women to men. Saguenay, Quebec had proportionately the most men.
You’ll find a map at the bottom of the article that lets you explore concentrations of single people by sex and age group, down to the census tract level.
But the part I found most fascinating was the availability by age chart. It shows all kinds of things, like that men are probably getting married earlier, that enough people get divorced in their forties for it to make a blip on the graph. And, of course, that the supply of available elderly women far outstrips that of men.
- Families with children 25 years and older at home - Average number of children per family - Common-law couples as a percentage of married couples - Female-headed single-parent families as a percentage of population - Divorced people as a percentage of population
One of my favourites shows the ratio of married to common-law couples in Ottawa/Hull:
Ever since the Ontario Auditor-General reported on the LCBO’s formula-based purchasing practices, criticizing a system that can result in one of the world’s largest alcohol buyers asking its suppliers to charge it more (the LCBO sets a target retail price for an item, which involves a fixed markup from the wholesale price; if the quoted price is too low, buyers can and do get the wholesale price raised to conform to the formula).
This seemed like an opportunity to test a theory I’ve had for a while that there are a number of public institutions in Ontario that have an underexploited, wide-open potential for data journalism, and that the LCBO is one of them.
I ended up filing two FOI requests with the LCBO. The first was aimed at trying to find out how much money had been left on the table in total through these wholesale price hikes. The response to this request didn’t yield any useful numbers, but did contain a list of purchases where the LCBO had asked for wholesale prices to be raised.
I switched strategies and asked for records of one transaction to see how it actually worked. The story is here.Basically, it explains how the LCBO asked for a price hike on a shipment of 180 cases of calvados intended for the Christmas market, rejecting a bid of $27,481 and asking to pay $31,351 instead to make the formula come out right. The documents are embedded in DocumentCloud at the bottom of the story.
The result was that population growth in the 905 tended to favour the provincial Liberals, at least enough to tip the balance in a notional Legislature using the new boundaries to a small two-seat majority, as opposed to the so-near-but-yet-so-far one-seat minority in the actually existing Legislature. The story is here.
In the aftermath of a high-profile shooting incident like the one at the PQ victory party Tuesday night in Montreal, it’s helpful to see what thegun registry databasewe obtained back in the summer has to say about the weapon allegedly used.
We filed an FOI request for EQAO results for the whole province as a way of avoiding having to sign a research agreement which was impractical for journalism projects.
The EQAO itself has come in for its share of criticism, as have standardized tests in general. My own view, for what it’s worth, is that the big patterns they reveal almost certainly have value, but that it would be unwise to get too wrapped up in the details.
Separate and public schools are mixed on the same map.
One thing I immediately noticed on the map screenshotted above was that the red schools, which fall below a given score in reading scores (percentage of pupils meeting or exceeding the provincial standard as an average of the available years) was that it had the classic check-mark shape of bad social indicators in Toronto: low incomes, homicides with male victims, certain kinds of disease, high school dropouts and on and on.
The same pattern also works in other Ontario cities: Hamilton, Windsor and Oshawa, as examples.
(There are exceptions: north-central Scarborough has census tracts with low median incomes but schools with fairly good test results, for example.)
To confirm what the map seemed to be showing, we created a new map superimposing the school results on another map (screenshotted above) showing median incomes from the 2006 census.
We confirmed the pattern we saw visually by using QGIS to place Toronto schools in their census tracts, then looked at average test performance in tracts by income band, then reversed the exercise by looking at average median income in the corresponding schools by test performance. The results were as follows:
With Dalton McGuinty’s government (tantalizingly or alarmingly, depending on your point of view) close to a majority, any by-election will be closely watched.
Bearing that in mind, here is an interactive map breaking down how Vaughan voted in the 2011 election. The top layer shows the plurality winner by poll; the next five show four parties in isolation, and turnout.
August 20: Updated to include Kitchener-Waterloo map: see below
October 6, 2011: How Vaughan voted
Use the scrolldown menu to see parties in isolation
October 6, 2011: How Kitchener-Waterloo voted
Use the scrolldown menu to see parties in isolation
A calendar-based heat map is a useful alternative to more traditional line graphs for showing change over time (it’s more original, it looks more interesting), so I decided to make one - by way of learning how to do it in Tableau - using our conception data.
(We crunched birth date data from B.C., Alberta, Ontario and Nova Scotia, deducting 40 weeks, to show an apparent spike in conceptions in the period roughly around Christmas.)
Searchable interactive: 93 years of Ontario baby names
by Patrick Cain
When we asked the Ontario vital statistics people for historical baby name data going back as far as electronic records were available, I thought we’d turn up 20 years or so - the equivalent Alberta data is available from 1990.
So it was a wonderful surprise when a spreadsheet popped up in my e-mail with totals for boys’ and girls’ names recorded in the province going back to 1917. The result is a pair of searchable interactives showing the ups and downs of thousands of Ontario baby names covering nearly a century.
Names come and go, but sometimes they reappear. The Ontario data shows that several girls’ names of the early 20th century, more or less disused for decades, are now being revived: Lillian, Hazel, Violet and Ruby.)
Edwards peaked in 1936, the year that the ill-fated Edward VIII was crowned - he would abdicate before the year was out, succeeded by his younger brother George VI, a better-respected king. (In Ontario, Georges kept declining after the coronation and also during the war, which seems somehow unfair.)
Charleses and Elizabeths both peaked in 1953, the year Elizabeth II was crowned. Elizabeth has another spike after the 1959 royal tour to Canada - a marathon one by modern standards, at 45 days.
Marilyns peaked in 1956. It’s hard to tell whether the main influence was actress Marilyn Monroe, then at the height of her career, or swimmer Marilyn Bell, whose arrival on the Canadian shore of Lake Ontario two years earlier had attracted hundreds of thousands of people.
Linda, a juggernaut name of the postwar baby boom, peaked in 1947 - Lindas had been rocketing upward in popularity as the Second World War progressed - and declined just as suddenly. Other classic postwar names are Donald (second peak in 1947) Barbara (peaks 1947), Sharon (peaks 1947) and Carol (peaks 1946).
Josephs, Marys and Maries peak in the late 50s and early 60s, then have a sharp decline apparently connected to Vatican II, the reform process that caused many changes in the Catholic church in that period. (Michael peaks in 1963, but with a gentler decline in following years.)
Here’s how it looks with a sample of names from various eras:
I blogged recently about an online application I built using Transport Canada’s Road Safety Recalls Database. The underlying data for my story is already available online, but was not easily available for download.
I called their media department to try to obtain a copy of the database that was powering their already public recall search tool, but was told that this was impossible. I ended up filing an ATIP request for the information. This is really overkill when the information is already public, just not in a convenient format… but it seemed the only way to get a copy of the database.
Description : This is an interactive web application built using Transport Canada’s Road Safety Recalls Database. It allows users to look up and compare recall information on different vehicles. This data was not on data.gc.ca. Although a searchable version of the database is on Transport Canada’s website, I was required to file an ATIP request for a copy of the data. I would argue that this data should be made available on data.gc.ca, as it would allow this kind of application to be constantly updated with the latest recall information, rather than just be a snapshot. There are many similar databases on GoC websites, which are public and searchable, but do not allow users to download the raw data. These should be added to the open data site.
The next day, I received this response:
Thank you for contacting the Government of Canada’s Open Data Portal.
We would like to thank you for bringing this application to our attention. We agree that this data, as well as its application, should be made available on the Open Data Portal.
After receiving your email this morning, we contacted Transport Canada to suggest that they provide the raw data on the portal. We anticipate that this dataset will be available on the Government of Canada’s Open Data Portal in the future.
Government of Canada
Based on this experience, I would suggest to people that they write the Open Data site and share their apps, and even suggest new datasets. It seems they read their email, and maybe we’ll start seeing useful data appearing on the site.
Crime has a schedule: Time-based visualization of 911 calls
by Patrick Cain
We’ve had a monstrous data set (over half a million lines) around for a very long time (since some point last summer) showing the time but not date of a year’s worth of Toronto police dispatch calls by category - ‘assault in progress and so forth’. The reason we never did anything with it had to do with not seeing how to visualize it in the way you see below. (The concept owed a lot to this graphic).
Recently, we figured out how to create what we needed in Tableau (see below) and what follows is a unique portraitof the daily cycles of Toronto’s dark side. Click on the image to see the graphic/story package:
I recently put together an interactive feature that lets users explore vehicle recall data and compare recalls on different cars. The full news story is here, with a nice TV report by Global Toronto’s Sean O’Shea.
The data for this is drawn from Transport Canada’s Road Safety Recalls Database, a searchable version of which is available on their website, although I had to do an Access to Information request to get the underlying data.
What excited me about this project is that it’s truly interactive. What you see on the screen completely changes depending on what you type. Everyone will have a different experience because everyone will want to search and compare different vehicles. Hopefully, it’s both informative and fun.
I used Tableau Public to power this feature. I’m liking this tool a lot lately because of the way you can link a variety of different visualizations together to really explore a given topic.
You do this by creating filters and applying them to multiple charts - in this case, I created some “Quick filters” and set them to “Global.”
It’s not perfect though. Tableau is powerful, but very non-intuitive to use. This project took a lot of trial and error and email exchanges with Tableau staff to put together. The Public free version also has a limit of 100,000 rows, which meant that I had to summarize my data and make it show only recalls from 1990 and later.
Still, pretty fun. Last week, I also did a similar visualization (under the hood at least) on MP’s expenses.
The story lends itself very well to a jQuery data table, an embeddable, searchable, sortable sort of spreadsheet I’ve been wanting to use since seeing it demonstrated at the NICAR conference. There was, as always, a learning curve - troubleshooting interactive tools on deadline (while also trying to report a story conventionally) isn’t anybody’s favourite experience, certainly not mine, but it worked out in the end.
- The first had to do with a quirk of the code in which the last item in the table must not end in a comma, as all the others do. Neglecting this breaks the table in a way that is only visible in IE - in other words, if you tested the thing in Firefox or Chrome at a low-stress point in the day, and didn’t see a problem then, there’s potential for a nasty surprise down the road at a high-stress point of the day. (NICAR-L was helpful solving this problem, I think within 15 minutes.)
- The second had to do with figuring out how to massage the complete table below 920px/w for display in our CMS. This turned out to be a problem with a very low-tech solution - column width is determined by the longest continuous string of characters in the column. A few instances of HARRINGTON&RICHARDSON instead of HARRINGTON & RICHARDSON will push a complex table too far out to the right. The solution, which does not really seem like a digital-age thing, involves holding a straight-edge (a pencil will do) up to the end of the line and scanning down for the long string that breaks the table. This looks ridiculous but works, like some other things.
Handguns (and a few automatic weapons) in the long gun registry
by Patrick Cain
I’ve been poring over a redacted copy of the national firearms registry we obtained under ATI recently.
The idea of asking for firearms data broken down by individual weapon and two-character postal code owes quite a lot to a project the Ottawa Citizen published a couple of years ago (though we’ve taken our analysis in another direction). With the federal government moving to delete the long gun portion of the registry, it was obviously our only chance to request this kind of data, so we did.
One thing that popped out of the data very quickly was that there were many, many handguns listed as non-restricted weapons, which means, all other things being equal, that they will be deleted along with the mainstream rifles and shotguns in the long gun registry. (The registry itself has not yet been deleted, but the data outside Quebec can no longer be accessed by police.)
The full database has 7.9 million weapons, which sounds intimidating, but I sliced out the non-restricted weapons with a category of “other” (as opposed to “rifle” or “shotgun” and so on), which reduced it to about 32,000, and looked at them in Excel. Weapons like air rifles and black powder muskets were fairly easy to screen out, which reduced the pool further.
The stray handguns are presented in a jQuery table, something that owes a lot to Chris Schnaars’ presentation at the NICAR conference. NICAR-L was helpful with a last-minute emergency (a bug in the code that I never would have found on my own.).
The fate of Quebec’s long gun registry is at stake this week in a Montreal courtroom, as the province’s lawyers argue that they should be able to use the Quebec data from the doomed federal long gun registry as the basis for a provincial one.
We recently obtained a redacted copy of the national firearms registry under ATI, made before the long gun data started to be deleted. Our first use of it is an interactive explainer looking at Quebec’s 1.6 million long guns (click on the image):
We’re working with a boundary set that covers the whole province, so it was easy enough to generate views of Hamilton and Ottawa as well.
Gonorrhea and chlamydia are common in the broader downtown, in a part of the city’s northwest stretching north along Jane St., Brampton, and eastern Scarborough.
Syphilis and HIV have very similar patterns to each other both tightly concentrated in the east downtown. The data itself came from a redacted version of the Ontario infectious disease database obtained under FOI.
We got the idea to show not just the beeyard locations, but also the foraging areas of those bees, so that people would know where they are likely to encounter bees from a given hive. It seemed simple enough: find out the range of a bee (about 3km) and draw circles with that radius around each point.
Finding an effective way to do this was surprisingly difficult. We hit upon this great tool from freemaptools.com, which allowed you to set a radius and bulk upload a set of points. It then draws a circle around each point and lets you export the resulting polygon.
It’s very easy to use and something we’ll be keeping in our toolbox for future projects.
We were busy Tuesday pumping out maps from the latest census release, which covered age and sex - too busy to blog about it until now.
The system we use is based on a KML file of census tracts in Fusion Tables - while the map series presents as 15 local maps, it’s actually 15 local views of one national map. The advantage of this system is that we only have to upload one map per subject, but (for example) Winnipeg can be sent the URL for Winnipeg’s maps, and so forth. The eightnow 12 new maps mostly focus on specific demographic slices, like over-65s, and gender ratios.
(Below is a Toronto-area map showing the percentage of children aged 0-9.)
PDFs are a fact of life in data journalism. Most of the time, when you request “an electronic file” or “an electronic document” in your Freedom of Information request, the end result is a PDF.
While it’s a step up from simply getting a stack of papers as your response, a PDF response is annoying in a number of ways. It’s hard to work with. Unlike a spreadsheet format, (Excel, .CSV, etc.) you can’t analyze a PDF. You often can’t copy and paste or export the data – sometimes Acrobat won’t even recognize the document as text!
Some departments do respond with an Excel file. Some are even nice enough, when I ask, to email me the Excel version of the PDF response that they had previously sent me. They don’t have to do this, and I really appreciate it.
But it seems like the default electronic file format is the PDF, which means that I will spend hours trying to force the information into a friendlier format. It doesn’t stop me from doing the story, it just makes it more difficult.
So why do many departments seem to favour the PDF? I decided to ask the Treasury Board Secretariat, the body charged with administering the federal government’s Access to Information legislation.
Here’s what they said.
I would like to know why, when a requester requests an electronic document, the response is usually provided as a PDF.
Why do departments seem to prefer releasing information as PDFs instead of a more open electronic file format, such as an Excel spreadsheet? This is particularly relevant in the case of a request for information from a database, which since it’s a table filled with numbers, would be more useful to a journalist in an Excel or other format.
Our government is committed to openness and transparency which is why we are pursuing the Open Government initiative that will continue to make government data freely available, and currently requires all completed ATI summaries to be posted online within 30 calendar days of being readied. Current Access to Information regulations direct departments to provide information in the format requested wherever possible, and our government continues to update and add to the already hundreds of thousands of data sets and the amount of information available to Canadians online in various formats. Where alternative formats are not available or suitable, the government will respond with a pdf version in order to ensure that requests for information are still carried out effectively.
So it seems all you need to do is ask – very specifically. It’s a valuable lesson. Next time, I will make sure to ask for a .CSV, and see what happens.
A TV/online interactive package aired Monday on the growing number of tuberculosis cases in the GTA, and the strong neighbourhood patterns shown by the patients’ postal codes.
In general, this kind of map is straightforward to do from a province’s reportable disease database - the disease and the first three characters of the postal code of the patient are generally releasable (at least in Ontario - Alberta refused to release similar data). You have to keep an eye on the proportion of patients whose postal codes are reported, but in this case we have FSAs for all but 21 of 632 Ontario 2010 TB cases, which seems more than acceptable.
Click on the map to see the story:
Here’s how it looked on TV (click the image to see the video):
We put together a quick-hit sort of map on Wednesday afternoon.
The Office of the Independent Police Review Director released a report on the conduct of police officers during the G20. In that report was a list of incidents where police officers stopped and searched individuals around downtown Toronto.
Since the report contained locations for each incident, we decided to map them, with the caveat that it is definitely not a comprehensive list of all the stops and searches that happened in Toronto that weekend.
You can see that there were a lot of incidents along Bloor Street, in Parkdale and the east end - well away from the downtown security fence.
Before legislation to abolish the registry was passed, Global News obtained postal code data for firearms owners from the RCMP using access-to-information laws. The maps below paint a picture of the geography of gun culture in Ontario.
The province-wide map shows low gun ownership rates in urban and suburban areas (Ottawa, the GTA, Hamilton-Niagara), higher rates in surrounding rural areas, and the highest ones, roughly, north of where viable farmland gives out.
A map showing only Toronto (scroll down to the bottom of the page) shows almost no gun licence holders downtown and the north half of Scarborough, and concentrations of gun owners - by Toronto standards - in Downsview and southern Etobicoke.
The story itself was based on a map package using the first three characters of the postal codes of registered organ donors in Nova Scotia and New Brunswick, which showed a lot of local variation. Nova Scotia does better than New Brunswick for organ donation, but New Brunswick does better than other provinces like Ontario and British Columbia.
Longtime Progressive Conservative MPP Elizabeth Witmer is resigning as MPP for Kitchener-Waterloo, leaving the door open to a snap byelection which could shift the balance of power at Queen’s Park.
In the October 6, 2011 election, Witmer won the seat with 43.4 per cent of the vote, defeating her Liberal opponent by about 7 per cent.
Our map shows the poll-by-poll results for the riding in the 2011 provincial election. Use the dropdown menu to see the PC, NDP, Green or Liberal votes in isolation.
In 2011, the PCs took the riding with the suburban outskirts of Waterloo, while the Liberals held a solid block of polls in the city centre. The NDP won in a few polls in the south of the riding. Click on the image to see the full interactive.
We’ve mapped marijuana grow operations in Toronto before, but this year’s iteration is more ambitious:
- First, we took the fresh 2011 census data and our census tract boundary set, mashed that up with several years’ worth of grow operation addresses, and created a map showing grow house rate by census tract. If you have the ability to count points within a polygon (Leslie showed me how to do this) it opens the door to interesting off-the-grid uses for census data.
Public Safety Minister Vic Toews is expected to announce announced today the closure of two federal prisons: the Leclerc Institution near Montreal and the venerable Kingston Penitentiary, which opened in 1835. Leslie and I mapped all of Canada’s federal prisons by way of an interactive explainer.
Even after Kingston Pen’s closure, three pre-1914 penitentiaries will still be in service: Dorchester Institution (1880), Stony Mountain Institution (1887) and the Saskatchewan Penitentiary (1911).
Neat new interactive feature from USA Today: Ghost Factories.
It’s an investigation of old smelting factory sites across the USA. I’m impressed by the investigation, the map, and even with how after you click on an individual site, you can tweet or Facebook it to your followers.
Welcome to the Data Desk blog, home of Global News’ Specials and Interactives team.
You’ll find posts here by Global News online journalists Patrick Cain, Leslie Young, Keith Robinson and others on occasion. We write about the craft of data journalism, Freedom of Information, and the technical side of online reporting tools.
In our daily jobs, we put together maps, visualizations and experiment with other fun ways to tell a story online. You can explore our collected work on the GlobalNews.ca site.
Most of the information we use to create our interactive features comes through Freedom of Information requests. We’ve sent out around 150 requests to over 50 government agencies in the past year. So you’ll find the occasional post about that here too.
Keep checking back for a behind-the-scenes look at how we do our data journalism. And of course, feel free to follow and contact us on Twitter: @globaldatadesk