The Economist reports on a paper that claims that
headline-grabbing scientific reports are the most likely to turn out to be wrong.
Given that the paper that includes this claim has done a pretty good job of grabbing headlines itself, should we expect that it will turn out to be wrong?
More seriously, the paper suggests that phenomena such as the winner’s curse may be having a deleterious effect on the publishing of research results.
The trial games of the 2008 TAC Market Design (or CAT) Tournament have now been completed, and eleven teams have qualified for the finals. These successful teams are:
BazarganZebel (Iran)
CrocodileAgent (Croatia)
DOG (UK)
Hairball (USA)
IAMwildCAT (UK)
jackaroo (Australia)
MANX (USA)
Mertacor (Greece)
MyFuzzy (Egypt)
PersianCAT (Iran)
PSUCAT (USA)
Congratulations to all these teams. The finals will be held in Chicago, IL on Monday 14 through Wednesday 16 July 2008, and you will be able to watch the live results here.
The New Yorker magazine recently carried a review by Louis Menand of a book by an economist, Bryan Caplan, “The Myth of the Rational Voter: Why Democracies Choose Bad Policies” (2006), a book which argues that most people are acting “irrationally” when it comes to certain questions of public policy.
There is a nice response to Caplan in a letter to the magazine by Christopher and Miranda Meyer of Lexington, Mass:
“Louis Menand neatly refutes Bryan Caplan’s argument that greater levels of voter participation are harmful, but he could have gone farther (Books, July 9th and 16th). Traditional economists insist that the failure of behavior to conform to their models is the fault of individuals, and that actions whose benefits escape the economic calculus are “irrational”. But understanding why people behave as they do teaches us more than declaring them wrong does. The thinking of most economists is limited by what they choose to measure – tangible economic value. They set about maximizing value, commonly expressed as gross domestic product, by optimizing resource allocation. Economists call this the “science of scarcity” but what if GDP is not scarce? Menand suggests that people value choice, in addition to goods and services. Perhaps justice, stability, diversity, and other social intangibles are, like choice, very scarce indeed, and highly valued. South Africans can feel the intangible return on their investment in Truth and Reconciliation, even if economists value only the costs. People vote because they feel that democracy plays a role in delivering these “uneconomic” valuables.”
According to a report in the FT, UBS is planning to launch a new automated trading fund for [tag]commodities[/tag].
“The UBS Commodities Portfolio Algorithmic Strategy System aims to fill a gap in the market between the long-only passive indices, such as the popular S&P GSCI – which has about $70bn tracking it – and services provided by commodities trading advisers and hedge funds.Peter Ghavami, UBS head of commodities, said: “As familiarity with the commodities asset class grows, an increasing number of investors are recognizing the value of taking a more active approach.”
The UBS Comm-PASS consists of a basket of strategies on 19 commodities future markets, with a 51 per cent exposure to energy.
The system, according to the bank, picks up on the momentum in the commodity markets and exploits it through automated long and short strategies.
. . .
Traditional commodities vehicles bet only on higher prices and usually roll over their positions every month, selling the front-month contract and buying the following contract.
The strategy results in losses when prices fall from one month to the next or during a bear market.
Sharp losses that resulted from this type of investment in late 2005 and in 2006 have prompted some institutional investors to consider more complex commodity investment strategies.”
In Vancouver for the finals of the TAC Market Design or CAT Tournament, being held at AAAI 2007.
By coincidence, the weekend edition of the National Post newspaper published yesterday a feature story on [tag]algorithmic equity trading[/tag] (which seems to be hidden behind a pay subscription wall, and not even accessible via the paper’s search engine*), and a related article on Canada’s emerging market for [tag]alternative trading[/tag] of equities, alternative trading systems, or so-called “[tag]dark pools[/tag]“. The first article quotes Forefactor, a Toronto research firm, as saying that 36% of all equity trades originating in [tag]Canada[/tag] use some form of [tag]algorithmic execution[/tag].
The CAT Tournament starts tomorrow, Monday 23 July 2007, at 0830 American Pacific time. You’ll be able to watch the game progress here.
* Some newspaper executives still don’t get it. Even if the Post article has to be paid-for to be read, at least let us see that the article exists online via your search engine! Or is it the case that an article on software trades does not even exist online?
A revised version of the CAT Tournament software platform and documentation has been uploaded onto Sourceforge. This version fixes some bugs identified in earlier versions and adds the assessment algorithms, and is available from here:
http://sourceforge.net/project/showfiles.php?group_id=170168
The additional documentation includes an overview of the game and its operations, available from here:
http://sourceforge.net/project/showfiles.php?group_id=170168&package_id=233340
or from here:
http://www.csc.liv.ac.uk/research/techreports/tr2007/tr07006abs.html
We had originally planned to hold a public trial run of the game during May 2007. For various reasons, we have now decided not to hold this trial. We are currently engaged in software testing of the tournament platform.
Further information will be posted soon about the qualifying rounds of the tournament.
We have finalized the assessment process for entrants to the CAT Tournament. The details will shortly be available in a document accessible from the JCAT sourceforge pages.
The document is also available as a PDF file from here.
————————————————————————————
The details of this process are also presented here in text format:
CAT TOURNAMENT AT TAC 2007
ASSESSMENT PROCESS
PUBLIC RELEASE DATE: 2007-04-10
0. DOCUMENT METADATA
The document presents the process for assessment of entries to the CAT Tournament to be held as part of the Trading Agent Competition (TAC) at AAAI 2007 in Vancouver, Canada in July 2007. The process presented here will be tested during the CAT game trial in spring 2007, and may be modified in the light of the trial experience. Registered entrants will be informed of any modification to the assessment and to the game following the trial.
This document and its contents are copyright © 2007 by the MBC Project CAT Tournament team.
1. ASSESSMENT PRINCIPLES
The assessment system has been designed to meet the following desiderata:
and
2. OUTLINE
The core idea is that entrants will be assessed on multiple criteria, which will be evaluated on a number of trading days. The assessment criteria to be used are described below in Section 4. In order to avoid effects arising from the fact that the Tournament has a start-day and an end-day, not all the trading days will be used for assessment purposes. The process by which days will be selected is described in Section 3 below.
3. SAMPLING METHOD
Step 1: We choose a random starting day, and a random ending day. Both selections are made prior to the commencement of the game, and remain secret
(from both game entrants and game organizers). The starting day will only be revealed at some point after it occurs. The ending day will only be revealed after completion of the game.
Step 2: Prior to game commencement, we also randomly choose days between the starting day and the ending day, on which assessment will be undertaken. These days are called “Assessment Days”. These selections also remain secret, and are made public one-by-one, at the end of each such selected day. In order to avoid manipulation, revelation that the assessment process has commenced may not be made until after the first several Assessment Days have occurred.
Step 3: At the end of each Assessment Day following revelation that the assessment process has commenced, the scores of each specialist against each criterion, and their total score, will be made public.
4. CRITERIA
It is planned to assess each specialist on each of the following criteria on each
Assessment Day:
Criteria will be normalized and then weighted equally (i.e., one-third each) to produce a score for each specialist for the Assessment Day. Scores will then be summed across all Assessment Days to produce a final game score for each specialist. The specialist with the highest final game score will be declared the winner of the tournament.
5. PRESENTATION of RESULTS
During the tournament, at the end of each trading day, we will present the values which each specialist achieves on each criterion, along with the cumulated values, and aggregated market statistics. In addition, on each Assessment Day, we will present separately the scores achieved by specialists. Thus, teams can monitor their performance (both absolutely and relatively) for each trading day, and for each Assessment Day. Revelation of the scores for Assessment Days will only be done once the fact that the starting day has passed is itself revealed.
—————————————————————————————
Ever wanted to design your own [tag]stock exchange[/tag]? Here’s your chance! We are organizing a new tournament at the [tag]Trading Agent Competition[/tag] in 2007, the [tag]CAT Tournament[/tag], which will allow marketplaces to compete against each other for the business of traders.
TAC will shortly be issuing an official call for participation. In the meantime, we have now made available preliminary documentation on the CAT tournament, along with a version of the CAT Tournament platform software, to assist participants in preparation of their entries.
Further information is available from the CAT Tournament pages.