Analysis of survey data in a study published this year shows that most people underestimate energy usage differences between household appliances, modes of transportation and recycling materials. The researchers speculate that the 100 watt light bulb example serves as an anchor point for thinking about energy usage, and that people make insufficient adjustments from this baseline when estimating the energy consumed by other things.

Our estimates don’t scale

This graph from the paper summarizes the household appliance findings, which is where they focus the majority of their attention in the discussion:

Perceived vs. actual energy used and saved by devices and alternatives.

A few of these surprised me. Adjusting a washing machine’s setting (presumably to use only cold water?) can save about the same amount of energy than it takes to run a central air conditioner, which in turn uses about the same amount of energy as a clothes dryer (measured in watt hours). Laptops consume significantly less energy than desktops. A space heater and a room air conditioner use about the same amount of energy, but a dishwasher uses a lot more. Note that the scale is logarithmic, increasing by a factor of ten with each tick.

Lack of incentive

All this got me thinking about what would motivate people to understand appliance power differences better, and adjust their appliance usage or purchasing behaviour accordingly. If you’re interested, like I was, to know more about the energy consumption of household appliances, Michael Bluejay’s site (I keep reading this as “Michael Bluth”) is a great place to start as he provides a good overview as well as a lot of up-to-date links to more specific information. But all this requires a fair bit of effort and just knowing that you may have your estimates wrong is scant incentive to learn how much it’s costing you to run the dishwasher several times a week, or to wash in warm water.

The authors of the paper say that, when asked how they could use less energy, survey respondents are more likely to think of curtailment behaviours (merely cutting back) rather than switching to more energy efficient devices, probably because the latter requires spending money now. It wasn’t clear to me at first why curtailment is considered much less effective than what amounts to buying more stuff, especially at a time when people are lucky just to have a job. One of the researchers explains elsewhere that curtailment is a problem of declining incentive over time: remembering to repeat the cutback activities versus buying appliances that are able to do the cutting back for you.

Rebound effect?

What about the “rebound effect?” I have read or listened to criticisms of efficiency recommendations that claim that if consumers adopt technologies that use less energy they will use those technologies more. As if on cue, one of the items that caught my eye on Bluejay’s site was mentioned in a comparison table: the very small energy consumption of LED night lights (0.5 W) compared with regular ones (5W). Thinking of the incandescent night light we use I checked out LED alternatives on Amazon, and the top reviewer for this popular product liked them so much he got one for every room.

Seems apt. But the reviewer didn’t mention that efficiency was important to him (though other reviewers do). One can think of reasons why this might have nothing to do with a “rebound.” Maybe he was looking for a light for every room and just liked these ones. Or maybe he chose energy-efficient LEDs because he wanted to use so many, rather than the other way around.

So is this effect real? Maybe, but it might not be big enough to worry about. According to Climate Progress, for example, evidence-based answers to this question are hard to find because of the many confounding factors in accounting for rebound behaviour. The aforelinked post describes an analysis of driver behavior in the US over the period 1966–2001, where better fuel economy was correlated with a modest increase in driving. But significantly, the authors report that the magnitude of the effect declines over time, and also with income level. It doesn’t take much to imagine that with the current economy being what it is, another oil price spike like 2008’s will suppress any rebound effect related to driving that compact car you got out of Cash for Clunkers.

Measure it

Since the paper about how people estimate their energy consumption was framed as a problem of individual choices in daily life it makes sense to look at what would motivate people—not just assist them but actually drive them—to make better choices, possibly (or probably) by educating themselves as a means of saving money (i.e. where saving energy is not the focus). In the case of home appliances, if you are “merely” motivated to save energy then measuring your usage is a way to test what works and know for sure if you are using less power/saving money overall (despite whether some of your usage behaviours rebound a little).

By way of anecdote, my Dad’s car estimates for him what his range is based on his driving behaviour. Not only does he like this, he treats it as a game to see how far he can drive on a tank. The numbers go up and down in real time, and it’s easy to associate this feedback with efforts at control such as maintaining a constant speed, lower revs, and fewer hard stops. There is maybe a five second lag after a throttle change and the dashboard displays an updated number. Gasoline prices in Australia are not as high as those in Europe, but they are significantly above the US, and could be seen as glimpse at our future here.

I just read a fascinating article in Home Energy Magazine from 1993 on Bob Hubbard’s new-at-the-time, off-the-grid, PV-powered house in Arizona. At the time that he built this house, Hubbard worked in the solar energy industry, so he had a comprehensive understanding of how to build efficiency in, and designed the building to use about a tenth of the energy consumed by a typical American home: about 855 kWh compared to the regional average of 9,300 kWh.^† (It’s big too, at 2,600 ft².)

What became clear as I read the story was just how important it was that Hubbard was able to measure the effect of each of his appliances and systems, both at the planning stages and after he moved in: from assessing the baseload of the house when nothing was being actively used, to knowing which time of day was best to use power tools, and even to determining that the gas oven had a hidden power cost while it was running that was not covered in the manufacturer’s specifications.

Software can help

All of this was possible because of the continuous monitoring and feedback of energy usage that Hubbard designed into his house. At the time it was built the entire residence was completely off-grid, and generated much more electricity than the two occupants used. The article suggests that if the house had instead been connected to the electricity grid his monthly bill from the utility company would have been about $7. Almost twenty years later, this is exactly the area where software could make a mark and help the general public to reduce their living costs.

Here’s an example of how real-time feedback makes a difference based on the writer’s experience with a “moment-of-use” monitor (in this case a PowerCost device):

A few minutes after one of my children started showering, the display unit jumped from its 3–400 watt idle value to over 1.5 kilowatts. […] I realized that was from the hot water heater turning on. When I hear the AC relay flip and the house begins to cool off, I watch the display unit jump to over 3kw. I found this type of monitor to be an effective way to begin understanding the systems in my home and how different behaviors affect how much energy we use.

This is a great example of the immediate usefulness of being able to receive feedback about your energy consumption, though it probably falls into the same camp as curtailment behaviours in that it takes a bout of motivation on behalf of the individual and is not easily sustainable even if you are a geek about it. The most well-known moment-of-use device is the Kill-A-Watt, which is affordable and easy to use, but only allows you to test devices that plug in to a wall socket. Even with the PowerCost which attaches at the breaker box, there is no data logging, so you have to be paying attention to the device to get any useful information out of it.

The web makes it awesome

It’s in web-based monitoring that the real leap forward lies:

Electric utility partnership with Google PowerMeter

Currently this is available almost nowhere. Obviously (well, it seems obvious to me, but I could be wrong) this is awesome, but it won’t take off until it’s something you can order through your utility company, and ideally would allow integration of different energy sources (e.g. electricity, natural gas, user-uploaded data on wood, pellets, propane, etc.). However, the good news is that if you are super motivated you can get started on this right now by purchasing a compatible device, or—if you have mad phreaking skillz—by making your own using open source hardware and software.

Google PowerMeter is a Google.org project, and doesn’t cost anything. (I’m not saying that there is therefore “no cost.” There are privacy implications, though from what I have read the status of this product means that Google is not intending to try to make money out of it, directly at least.) What it does do is makes it trivial to include energy monitoring in your life the same way that you check your bank balance online. And it allows you to represent the data in different ways and compare your usage history with data from the general population.

I suppose this is what people who use the term “smart grid” are talking about, or maybe the idea of combining something like this with appliance-level communication within the home, which—provided it can all be done without using too much power—would make it much easier than it currently is to (a) have a clue, and (b) care about how much energy you consume, in the home and beyond.

Update: a promising development.

† This was in 1993. As of March, 2010 the national average is a rather shocking 11,040 kWh (with regional averages ranging from 6,252 kWh in Maine to 15,624 kWh in Tennessee).

Related posts:

• Microwave, macro annoying
• Renewable energy at home
• The future of advertising

The eighth rule of Facebook is, verbatim: if you don’t need to use Facebook for work then walk away soldier, and don’t look back, even for a second. For everyone else, here are some tips for protecting your privacy and information security on The Social Network.

First up is some background information in the form of a rant, with more movie references. If you just want the “how to” details, then skip to Global Privacy Settings.

Facebook wants you to over-share

Over the past five years Facebook has gradually changed it’s default privacy settings for both new users and new features. If you join in 2010 and do not change your default privacy settings then your activity can be observed by a lot more people than if you had done the same thing in 2005. Here is a nice set of pie charts that illustrates the story.

Similarly, if you joined in 2005 and have never changed your privacy settings then your activity has gradually become available to a larger number of people because Facebook has added new features that don’t respect or relate to your existing privacy settings. Places, discussed below, is a good example of this.

Two levels of concern

Given the creeping exposure of all manner of personal data over time, there are two levels of concern with Facebook. The first is that your activity may be observed by undesirables (employers, children, parents, stalkers, etc.): the privacy problem. You want to be sure that you configure and use your Facebook account in such a way that your disclosed data can only be observed by the people you choose. The easiest way to deal with this is to treat everything you do on Facebook as potentially a public statement. Another way is to use friend lists.

The second level of concern is that even with good privacy controls that protect you from other individuals on Facebook, your activity over time may generate a behavioural profile that can be used to identify you even if your name never appears in the data comprising that profile. This is exacerbated by factoring in your activity on other social networks. Facebook, its advertising partners or, potentially, law enforcement or hackers have access to aggregate data that could be used to de-anonymise you: the information security problem.

Evil fireball

Just like the evil space fireball in The Fifth Element, that only grows bigger when General Staedert orders his crew to fire at it, Facebook gobbles data and the more active you are, the more it knows about you. Example: deleting a photo you don’t like equals more data about you (she didn’t want that photo to be seen by her friends). Deletions, by the way, are recorded. The data that is deleted remains recorded, etc. Everything is saved. Evil fireball.

Of course, jerks who see the world in black and white because it suits their agenda will tell you that if you have done nothing wrong ever in your entire life, and know that you never will, then you have absolutely nothing to hide from the fireball. They do not care about your privacy and liberty, they just want you to get out of their way. Nevertheless: the best way to deal with these problems is not to join Facebook. The second best way is to treat everything as public even if you have taken measures to control who sees it.

Global privacy settings

On May 26 Facebook replaced it’s convoluted privacy controls with a unified and simplified global settings panel. Nick O’Neill, who has written many useful articles on Facebook privacy, provides and overview in his post “Ten things about today’s privacy changes.” This is easiest way to start protecting your privacy.

Go to Account → Privacy Settings and choose something other than “Everyone” or “Recommended”. Facebook’s “Recommended” settings are, naturally, not very private. If you want my advice, choose “Friends Only” and lock that in first, then customise it further to restrict some of the settings to either a list, a specific group of names, or “Only Me.”

Note that at the top of the settings page there is a section called “Basic Directory Information”. Click on the unassuming little link that says “View settings” to both view and change your default directory settings. This is basically the information that people can find out about you through Facebook’s various search features.

At the bottom of the settings page there is also a section called “Applications and Websites”. Click on “Edit your settings” to set limits to the kinds of data that Facebook applications can access about you by default should you choose to install them (something that I recommend you avoid as much as possible).

Settings for ‘Places’

Following the popularity of location-based social networks like Foursquare and Gowalla, Facebook in August launched a location “check in” feature called Places that allows you to send your geographic coordinates to the system to let people know where you are. With all of their usual charm and tact they set up this feature to allow, by default, other people to check you in to places. (Bruce Schneier, whom I linked to a couple of times above, calls this type of information about you incidental data.)

Nick O’Neill again provides a good overview and some advice in “Places privacy settings.” The most important thing is probably to prevent other people from being able to check you in to places so be sure to set “Friends can check me in to Places” to “Disabled.”

Customising your settings

If you plan to use Facebook to say things that really do need to be restricted to a smaller group of people then you should customise your privacy settings. O’Neill’s post “Five privacy tips,” written before the privacy simplification changes in May, is still very useful in this regard, especially for the details it provides on using fine-grained Custom permissions, which is what you need to use if you want to restrict a data type to yourself or a short list of specific indiviuals.

Related to this is a change to the Publisher that went into effect this week, and relates to your everyday use of Facebook. The Publisher is the open text box at the top of your Wall and News Feed screens that invited you to start typing something (“What’s on your mind?”) has now been replaced with a Share bar that presents the following options: Status, Question, Photo, Link and Video.

When you one of these you get either a text box for typing, or a set of upload options, as well as a Permissions drop-down that allows you to limit who can see what you type or upload. Use the permissions feature to restrict your status updates to trusted friends if you are planning on sharing sensitive information.

Or, better yet, don’t share sensitive information on Facebook.

Related posts:

• Bad cookies
• Ramifications
• How to make a Facebook page

An excerpt on global web usage from the (newly redesigned) World Wide Web Foundation site, whence I also co-opted the graphic above:

Only 25% of the world’s population uses the Web, despite the fact that more than 70% (and growing) have access to mobile or fixed communications.

Useful to remember next time the “ubiquity of the web” argument rolls around. The WWW Foundation has a lot of good projects on the boil, of which the program to “teach youth in the most economically-challenged neighborhoods of major cities to develop Web applications” is especially cool.

Related posts:

• Joe Clark: Unreadable
• ‘Doing the whites’ in kWh
• Serious business

According to data provided on Google Flu Trends comparing to this year to 2003–2004 (a particularly severe year) the rate of estimated flu cases has climbed high and early. As for severity by state, this is what the map looked like in November last year:

Estimated USA flu severity map as at November 11, 2008 (Image: Google.)

And here’s what it looks like this year as of the end of October:

Estimated USA flu severity map as at October 30, 2009 (Image: Google.)

Finally, here’s a chart showing a comparison of Google estimated prevalence data and CDC prevalence data since the 2003–2004 season:

Yellow: CDC data. Blue: Google data. (Image: Google.)

How valid is this information (i.e. how much is it a reflection of actual rates of influenza infection)? It appears that Google’s flu prevalence estimate data based on search counts (a form of “syndromic surveillance”) are accurate predictors of reported rates of influenza in the United States. The Flu Trends About page (linked above) mentions the Nature study that Google undertook with a CDC researcher, and links to a free, Google-hosted version of the paper as a PDF file if you’re interested. This is further corroborated by the modest paper “More Diseases Tracked by Using Google Trends” that was published in the CDC web journal Emerging Infectious Diseases (Volume 15, Number 8–August 2009)^† though this is more of a research note than a detailed study.

I did find one interesting statement on this by the CDC, in answer to a direct question from a reporter. At a press briefing on May 5 Alice Park from Time posed the following to the CDC’s Richard Besser:

I just wanted to ask you about your opinion on some of the new flu tracking, surveillance service out there. I know that the CDC has worked with Google flu trends, for example, can you talk a little bit about how helpful that type of information is particularly now to get a better sense of the dynamics of the outbreak, you know, where it might be increasing or what the ebb and flow of it and is it getting worse, is it tapering off, can you comment a little bit about how useful those kind of methods are.

Besser replied:

In terms of our ability to detect emerging infectious disease, new infectious diseases, we’re constantly looking for what we call situational awareness. I mean what’s going on out in the communities. And we’re looking at you know, many, many different sources of information. The Google flu tracking information, there was a study done with Google in conjunction with CDC to look at can you use that information, can you use people going on the web to find information about flu as an indication of where flu is taking place? And the first year looking at that in terms of looking back, it was very helpful. The question is looking forward can you see that? As of two weeks ago, Google hits on flu on H1N1 are just off the charts. And so our website gets 8 million hits a day. So looking for a signal of increased activity on the web in a particular place isn’t very useful. But we’re open and are continually looking at various approaches to early detection because the sooner you can detect a problem, the sooner you can understand it and implement appropriate control measures.

Translation: I’m comfortable saying, with the benefit of hindsight, that Google Flu Trends in its first year of operation has done a pretty good job, but you are not going to hear me endorse it as a predictive tool for tracking H1N1. Or: it’s been shown to be valid, but we don’t yet know how reliable it is from year to year (or, for that matter, how much it might be influenced by special kinds of flu or fears about flu).

So, for what it’s worth, I’m going to go ahead and take it all with a grain of salt for now. I’ll say one thing though: it makes for some interesting graphics, and you have to admit that we’ve had a lot of graphs going down these past 18 months…

Related

The follow-up article for Time by Alice Park, “Is Google Any Help in Tracking an Epidemic?” published on May 6, 2009.

† Be warned: the CDC web site is even slower than your Toyota Yaris.

Related posts:

• Google Flu Trends
• ‘Doing the whites’ in kWh

The key issue is the cost of management. Thus far we focused only on computing resources such as storage, computation, and data transfer; but the cost of managing all of this—making sure the right software version is installed, that data is backed up, that spam filters are updated, and so on—is a significant part of the picture. Indeed, as the cost of computing resources, on both client and server sides, continues to fall rapidly, management becomes a bigger and bigger fraction of the total cost. And so we move toward an approach that minimizes management cost, even if that approach is relatively wasteful of computing resources. The key is not that we’re moving computation from client to server, but that we’re moving management to the server, where a team of experts can manage matters for many users.

This trend may be turning traditional technology services (from the perspective of IT departments) into what economists call “externalities.” The biggest risk for organisations here is probably going to be increased vulnerability to information security issues. The convenience of having everything available in a browser comes at the cost of requiring a network to accomplish basic tasks, and exposing ever more of those tasks to the Internet.

You change your password every few months, right? ;-)

Not that I really want to be under anybody’s microscope. But if anyone’s going to be analyzing my phone calls, payment transactions, emails and IMs, I’d rather it be researchers who will publish their findings, instead of secretive intelligence agencies. If our communications aren’t going to be private, let’s at least use these capabilities for clear, transparent public benefit.

Here’s an e-affirmative action proposal: For every intelligence agent that has access to mass surveillance data, one academic researcher should have access to the same information. And report on it.

At least then we’d know what the heck was going on.

Not ideal, but certainly pragmatic.

Related posts:

• Protecting your privacy on Facebook
• Paper trails

SuperDuper runs on Intel and Power PC Macs, and is compatible with Time Machine under Leopard. Download and use it forever for free, or buy for US $27.95 to unlock scheduling and Smart Update. (Smart Update copies only the data that has changed since your last backup, enabling you to backup your hard drive in minutes instead of hours. Which means you’ll actually get into the habit of running the backup program every day. Which means you’ll never lose your work. Best $27.95 you’ll ever spend.)

We got SuperDuper! after the PowerBook Catastrophe of 2005 and it saved Elena’s data from annihilation just over a year later when the replacement hard drive failed.^†

† This text was emphasised for your benefit.

Related posts:

• Regarding Ma.gnolia
• On reinstalling OS X Leopard
• Ramifications

As more newspapers go out of business in the coming years, I think it’s important that our sympathy for individual employees not translate into the fetishization of newsprint as a medium. And it’s especially important that we not confuse newsprint as a medium with journalism as a profession. Newsprint and journalism have been strongly associated in the past, but this an accident of technology, not something inherent to journalism. Journalism—the process of gathering, summarizing, and disseminating information about current events—has been greatly enriched by the Internet. Journalists have vastly more tools available for gathering the news, and much more flexible tools for disseminating it. The replacement of static newspapers with dynamic web pages is progress.

I agree (and I admit that it’s a lot easier to get philosophical when you’re livelihood is not directly dependent on the institutions that are foundering). It’s why Adrian Holovaty has, for some time, advocated a stronger connection between journalism and programming: the latter allows investigators to do the former in new ways. Machines can do a lot with information if that information is structured:

The goal for me, a data person focused more on the long term, is to store information in the most valuable format possible. The problem is particularly frustrating to explain because it’s not necessarily obvious; if you store everything on your Web site as a news article, the Web site is not necessarily hard to use. Rather, it’s a problem of lost opportunity. If all of your information is stored in the same “news article” bucket, you can’t easily pull out just the crimes and plot them on a map of the city. You can’t easily grab the events to create an event calendar. You end up settling on the least common denominator: a Web site that knows how to display one type of content, a big blob of text. That Web site cannot do the cool things that readers are beginning to expect.

Because the dominant medium is now computerised and networked, there needs to be some kind of convergence between old-fashioned muckraking and the production of structured data, if the organisations in which journalism thrived in the twentieth century are to live on in the twenty-first.

Related posts:

• What’s next for EveryBlock?
• The future of Canadian media
• Kids these days

Bruce Schneier on data as pollution:

Your future has no privacy, not because of some police-state governmental tendencies or corporate malfeasance, but because computers naturally produce data. […]

Just as we look back at the beginning of the previous century and shake our heads at how people could ignore the pollution they caused, future generations will look back at us—living in the early decades of the information age—and judge our solutions to the proliferation of data.

I’d add: and because the data they produce is good for making profits. He gives a lot of examples that illustrate just how impossible privacy has become. To have any, I suppose you have to be either extremely poor, extremely wealthy, or extremely isolated.

Related posts:

• SuperDo
• Protecting your privacy on Facebook
• The economics behind ‘cloud computing’

We have a number of ideas for sustaining our project beyond a dependency on grants, like building a local advertising engine and/or selling hosted versions of the open-source software, but we’re sure there are other ways for EveryBlock to be a successful business. That brings me to the reason I’m posting this—we’re looking for ideas and partners who would be interested in helping us figure this out. If you have any ideas or suggestions, get in touch with me. I’m confident we’ll make something happen; it’s just a matter of how.

EveryBlock’s core innovation is to take existing public information (civic and municipal data, journalism, photographs, user reviews, etc.), and to expose it on the web based on the physical locations to which that information applies. Among other things, EveryBlock has played a key role in opening up government data on the web; Daniel X. O’Neil:

At EveryBlock, where my main role is to work with municipal governments to uncover new data sets, we’re experimenting with a new form of journalism where we treat freshly updated public records as block-level news. It’s a big job to acquire ongoing feeds of government data, and we have a broader goal of spreading the gospel of open data.

If you have thoughts I urge you to share them with the development team. EveryBlock is a point of genuine innovation that has produced a useful new service, in a landscape increasingly littered with irrelevance.

Update: On August 17, 2009 Adrian Holovaty announced that EveryBlock has been acquired by MSNBC.com:

We’ll continue to run the first and best microlocal news Web site on the planet, with the same six people, with the same logo and design, with the same everyblock.com domain. MSNBC.com has hired our whole team, and they’ve made it clear to us that we’ll be driving the site’s strategy and implementation, and that our site will remain an independent destination as a community service.

Sounds like good news.

Related posts:

• Skewed data
• Google Flu Trends
• Flu trends comparisons

The data set for the competition consists of more than 100 million anonymous movie ratings, using a scale of one to five stars, made by 480,000 users for 17,770 movies. Note that the user-item data set for this problem is sparsely populated, with nearly 99% of user-item entries being zero. The distribution of movies per user is skewed. The median number of ratings per user is 93. About 10% of users rated 16 or fewer movies, while 25% of users rated 36 or fewer. Two users rated as many as 17,000 movies. Similarly, the ratings per movie are also skewed: almost half the user base rated one popular movie (Miss Congeniality^†); about 25% of movies had 190 or fewer ratings; and a handful of movies were rated fewer than 10 times.

So regarding users: there is rich data about atypical users, and sparse data about the majority. And regarding movies: there is copious information about popular movies, and little about the long tail.

† WTF?

Related posts:

• Netflix streaming video
• Protecting your privacy on Facebook
• What’s next for EveryBlock?

According to the bureau’s sampling and calculations done on that sampled data, America shed 598,000 non-farm jobs in January, worse than expected and boosting the unemployment rate to 7.6 per cent. Much has been made in the press about this being the highest number of jobs lost in 34 years. But no one in the press points out that to make such a comparison fairly you need to adjust the number of jobs lost against the size of the labour pool, which is much larger in the States than it was 34 years ago.

Similarly, business news organizations are always talking about how much the Dow Jones Industrial Average or the Financial Times Stock Exchange index drops by X or Y points, the most since Z, and they do not adjust that drop against the size of the index at the time, I guess to make a 300-point drop sound more dramatic than it really is. As an index or a labour pool grows, you would expect swings to be bigger in number. The point is this: It isn’t as bad as it sounds, even though it is bad.

Really? Hmmm… I’m sick of the TV news too.

There’s a point where, because we already know that things are bad, trying to quantify it is moot. We passed that point.

Related posts:

• A time of bad debts
• The top ten podcasts
• News in the time of Internet

The survey should take only about 20 [10 –Ed.] minutes or so to complete. It’s anonymous, but we do ask you register, both to help us maintain the integrity of the result, and just as importantly, to enable us to give away cool stuff to those who participate, including passes to future Web Directions events of your choice, including Web Directions North in Denver. Of course, we’ll only use any email addresses for the purposes of communicating about the survey, and won’t share them with anyone.

More data, better web, trip to Denver?

Related posts:

• The ALA web survey
• ‘Doing the whites’ in kWh

We have found a close relationship between how many people search for flu-related topics and how many people actually have flu symptoms. Of course, not every person who searches for “flu” is actually sick, but a pattern emerges when all the flu-related search queries from each state and region are added together. We compared our query counts with data from a surveillance system managed by the U.S. Centers for Disease Control and Prevention (CDC) and discovered that some search queries tend to be popular exactly when flu season is happening. By counting how often we see these search queries, we can estimate how much flu is circulating in various regions of the United States.

They claim that this gives results two weeks earlier than existing methods. I wonder if you could predict an election with data like that…

Edited to add: What a difference a day makes. This screenshot shows that several states have moved from “low” to “moderate” flu activity based on data from the past 24 hours:

Compare this to Lydia’s screenshot which shows how the map looked yesterday. “Moderate” now includes Vermont, West Virginia, Alabama, Georgia and South Carolina, and Nevada has moved from “minimal” to “low.” Here comes that spike.

Related posts:

• Flu trends comparisons
• Skewed data
• What’s next for EveryBlock?

Related posts:

• State of the Web 2008
• Ramifications
• ‘Doing the whites’ in kWh