Competitive Intelligence Marketplace

The STASI Reconstruction Case

Kent's Imperative beat me to the punch in writing about the Wired article on the effort to reconstruct shredded STASI documents that literally fill a warehouse.  It's interesting in a sociological sense, seeing how important it is to citizens of the former East Germany to see their STASI files, but it's also a technological marvel in how the reconstruction project is being implemented.  You can read the Wired article here, and I highly recommend it.

Here's what I consider a great observation from Kent's Imperative:

The STASI legacy has tainted subsequent generations of intelligence far removed from the same evils, but also serves as an instructive sort of anti-model of actions – and more importantly – an underlying intent, one that must be avoided by ethical professionals at all costs. Thankfully, the impulses that drove the STASI are quite alien to those which have developed in the American intelligence tradition – and we hope that this will remain so as long as the profession endures in an apolitical and accountable form.

Prediction Markets

I happened to come across two different mentions of prediction markets within about five minutes.  One is a Freakonomics blog post about several research papers related to prediction markets and the second a post on Kent's Imperative titled A Hard Look at Prediction Markets.  The second is a good read in and of itself and the first is a good source for links to prediction market related information.  Interestingly both reference the release of results from Google's internal prediction market.  From Kent's imperative:

The study covers nearly three years of the operation of an exchange which handled over 70,000 transactions – each conveying a degree of belief on one of almost 300 particular questions, on behalf of 1500 active employees (although nearly 6500 held accounts that were not used.) Interesting, they identify unexpected influences due to physical proximity, as well as the impact of cognitive bias towards optimism based on employee fiscal considerations created by Google’s rising share price. Also quite interesting was their observation that new employees were more influenced by this bias, and that staff with longer tenure within the firm tended towards more calibrated judgment – a not inconsistent phenomenon within any analytic activity.

Do Companies Lie?

An interesting post on Knowledge is Power titled "Do Companies Lie?" provides an example of why lying is often harder and less effective than simply saying "no comment":

Some researchers believe that mis-information is planted frequently; however, I have found that most sources will refuse to answer my questions rather than lie. Lying takes effort to create and maintain a creditable story with all the pieces hanging together.

However, recently I researched a private company that is deliberately mis-representing its revenue figures. During the late 1990’s, the management confirmed in articles in the local newspaper that revenue was around $125 million and spoke about advertising campaigns that ranged up to $20 million. But the most recent revenue figure that I could find in 2007 was $50 million. Certain information is public such as leasing new space with the help of local incentives and building permits to alter the space. So I also knew that this company had increased its manufacturing capacity by 50% two years ago by acquiring an empty plant with the help of state job creation incentives. What gives?

Turns out that an employee with an Arabic-sounding name in a key position at the firm had been arrested for terrorism by the FBI shortly after 9/11. The founder and CEO of the firm were very active in Jewish philanthropy at the time and this private firm decreased its already low profile to practically nothing after that incident.

Once I understood the firm’s situation, I threw away the $50 million figure and estimated that the real revenue for this organization to be $250 million.

Uclue

From Cool Tools comes this post about Uclue.  Here's an excerpt:

Google Answers was a great service I used and recommended. Sadly it was closed. Many of the free-lance researches from Google Answers moved to a new independently owned site, Uclue, that offers a similar service. You ask a question, announce a price you think an answer is worth, and if a top-notch researcher thinks your fee is fair, they will research your question. Questions can be quickies worth $5, or more complicated queries costing $200.

In my experience their answers are solid and reliable. You can always ask for clarifications. As with Google Answers, the results are public. That means it pays to search the site for previous similar questions. It also means that your answer won't be confidential. (Indeed. The answer to a question I commissioned on Uclue was Slashdotted.)

Asking and Answering the Right Questions

Arthur Weiss has an excellent post I won't even bother to summarize here.  Please just follow this link and read it.

And no I don't think it's excellent because he references this blog. I am pleased, however, to know that Arthur actually reads it.

Swarm Theory and Potential Impact on Business

National Geographic has a fascinating article on swarm theory and some of its potential for business and industry:

That's how swarm intelligence works: simple creatures following simple rules, each one acting on local information. No ant sees the big picture. No ant tells any other ant what to do. Some ant species may go about this with more sophistication than others. (Temnothorax albipennis, for example, can rate the quality of a potential nest site using multiple criteria.) But the bottom line, says Iain Couzin, a biologist at Oxford and Princeton Universities, is that no leadership is required. "Even complex behavior may be coordinated by relatively simple interactions," he says.

Inspired by the elegance of this idea, Marco Dorigo, a computer scientist at the Université Libre in Brussels, used his knowledge of ant behavior in 1991 to create mathematical procedures for solving particularly complex human problems, such as routing trucks, scheduling airlines, or guiding military robots.

In Houston, for example, a company named American Air Liquide has been using an ant-based strategy to manage a complex business problem. The company produces industrial and medical gases, mostly nitrogen, oxygen, and hydrogen, at about a hundred locations in the United States and delivers them to 6,000 sites, using pipelines, railcars, and 400 trucks. Deregulated power markets in some regions (the price of electricity changes every 15 minutes in parts of Texas) add yet another layer of complexity...

Ants had evolved an efficient method to find the best routes in their neighborhoods. Why not follow their example? So Air Liquide combined the ant approach with other artificial intelligence techniques to consider every permutation of plant scheduling, weather, and truck routing—millions of possible decisions and outcomes a day. Every night, forecasts of customer demand and manufacturing costs are fed into the model.

"It takes four hours to run, even with the biggest computers we have," Harper says. "But at six o'clock every morning we get a solution that says how we're going to manage our day."

For truck drivers, the new system took some getting used to. Instead of delivering gas from the plant closest to a customer, as they used to do, drivers were now asked to pick up shipments from whichever plant was making gas at the lowest delivered price, even if it was farther away.

"You want me to drive a hundred miles? To the drivers, it wasn't intuitive," Harper says. But for the company, the savings have been impressive. "It's huge. It's actually huge."

Fuld Says CI Spending Will Rise Ten-Fold by 2012

Fuld & Company has done a survey that shows CI spending will rise ten-fold by 2012:

According to a study conducted by research consultancy Fuld, America�s largest 1,000 firms are set to increase spending on competitive intelligence (CI) activities to at least $10 billion by 2012.

This prediction is based on data taken from the firm�s recent benchmarking survey of 141 mostly large Fortune 1000-class corporations, in which respondents provided budget ranges for their CI programmes.

The $1 billion currently being spent on CI includes internal staff costs and does not include expenditure associated with using external consultants, subscriptions, or database services. The firm�s President Leonard Fuld explained: �As a means of comparison, this number was close to zero only twenty years ago, when whatever monies spent in this arena were usually buried within the market research or perhaps strategic planning budgets.�

Doc Searls Takes Research Organizations to Task

Doc Searls penned an item on his blog titled "Researchers need to get some clues" in which he takes research organizations, MediaPost in particular, to task for their antiquated ways.  Here's an excerpt:

Seems to me that much of the research business, and the email bulletin business that oozes out of it, are both archaic beyond excuse.

There is so much good, hard, interesting and useful information out there, right now, syndicating itself to the world. Why not harvest some of that? Why not syndicate your own stuff, so you can join the growing syndisphere?

Advice: screw the email bullshit. And screw the promotional index pages. Start syndicating reports. Start blogging. Sure, keep issuing press releases for the shrinking population that still looks for clues in that clueless source. But get hip to what's happening in Live Web development.

And it that's not tempting enough, just follow the money.

Researchers Need to Get a Clue

Doc Searls takes issue with MediaPost and their crappy email report service.  He also stresses that research organizations need to focus on syndicating and blogging. Here's an excerpt:

 Seems to me that much of the research business, and the email bulletin business that oozes out of it, are both archaic beyond excuse.

There is so much good, hard, interesting and useful information out there, right now, syndicating itself to the world. Why not harvest some of that? Why not syndicate your own stuff, so you can join the growing syndisphere?

Advice: screw the email bullshit. And screw the promotional index pages. Start syndicating reports. Start blogging. Sure, keep issuing press releases for the shrinking population that still looks for clues in that clueless source. But get hip to what's happening in Live Web development.

And it that's not tempting enough, just follow the money.

Read it all here.