Thursday, July 28, 2016

Veracity, context, and relevance

From The Invention of News: How The World Came to Know About Itself by Andrew Pettegree.
The trouble with the newspapers was that they were not very enjoyable. Although it might be important to be seen to be a subscriber, and thus to have the social kudos of one who followed the world’s affairs, the early newspapers were not much fun to read. The desiccated sequence of bare, undecorated facts made them difficult to follow – sometimes, plainly baffling. What did it mean to be told that the Duke of Sessa had arrived in Florence, without knowing who he was or why he was there? Was this a good thing or a bad thing? For inexperienced news readers this was tough going. People who were used to the familiar ordered narrative of a news pamphlet found the style alienating.
We are again faced with this conundrum though now in an age of data plenty. The central issues remain veracity, context and relevance. Natural language processing and AI both help but we are still a long way from unlocking this issue.
The news reporting of the newspapers was very different, and utterly unfamiliar to those who had not previously been subscribers to the manuscript service. Each report was no more than a couple of sentences long. It offered no explanation, comment or commentary. Unlike a news pamphlet the reader did not know where this fitted in the narrative – or even whether what was reported would turn out to be important. This made for a very particular and quite demanding sort of news. The format offered inexperienced readers very little help. The most important story was seldom placed first; there were no headlines, and no illustrations. And because newspapers were offered on a subscription basis, readers were expected to follow events from issue to issue; this was time-consuming, expensive and rather wearing.

The World Bank challenge

Heh. Reminds me of A Void by Georges Perec. He wrote the entire three hundred page novel without ever using the letter "e".

All this time, a second type of fungus has been hiding in plain view.

A great Example of how, despite what the anti-science crowd claim, the science is never settled. From How a Guy From a Montana Trailer Park Overturned 150 Years of Biology by Ed Yong.
In 1995, if you had told Toby Spribille that he’d eventually overthrow a scientific idea that’s been the stuff of textbooks for 150 years, he would have laughed at you. Back then, his life seemed constrained to a very different path. He was raised in a Montana trailer park, and home-schooled by what he now describes as a “fundamentalist cult.” At a young age, he fell in love with science, but had no way of feeding that love. He longed to break away from his roots and get a proper education.


Lichens have an important place in biology. In the 1860s, scientists thought that they were plants. But in 1868, a Swiss botanist named Simon Schwendener revealed that they’re composite organisms, consisting of fungi that live in partnership with microscopic algae. This “dual hypothesis” was met with indignation: it went against the impetus to put living things in clear and discrete buckets. The backlash only collapsed when Schwendener and others, with good microscopes and careful hands, managed to tease the two partners apart.

Schwendener wrongly thought that the fungus had “enslaved” the alga, but others showed that the two cooperate. The alga uses sunlight to make nutrients for the fungus, while the fungus provides minerals, water, and shelter. This kind of mutually beneficial relationship was unheard of, and required a new word. Two Germans, Albert Frank and Anton de Bary, provided the perfect one—symbiosis, from the Greek for ‘together’ and ‘living’.


In the 150 years since Schwendener, biologists have tried in vain to grow lichens in laboratories. Whenever they artificially united the fungus and the alga, the two partners would never fully recreate their natural structures. It was as if something was missing—and Spribille might have discovered it.

He has shown that largest and most species-rich group of lichens are not alliances between two organisms, as every scientist since Schwendener has claimed. Instead, they’re alliances between three. All this time, a second type of fungus has been hiding in plain view.
Read the whole thing, it's an interesting story at several levels.

There are more mysteries than we acknowledge and no matter how high a degree of consensus, science has a way of over-turning the apple-cart.

Wednesday, July 27, 2016

From The Invention of News: How The World Came to Know About Itself by Andrew Pettegree.
The printed news pamphlets of the sixteenth century were a milestone in the development of the news market, but they further complicated issues of truth and veracity. Competing for limited disposable cash among a less wealthy class of reader, the purveyors of the news pamphlets had a clear incentive to make these accounts as lively as possible. This raised real questions as to their reliability. How could a news report possibly be trusted if the author exaggerated to increase its commercial appeal?

The emergence of the newspaper in the early seventeenth century represents an attempt to square this circle. As the apparatus of government grew in Europe’s new nation states, the number of those who needed to keep abreast of the news also increased exponentially. In 1605 one enterprising German stationer thought he could meet this demand by mechanising his existing manuscript newsletter service. This was the birth of the newspaper: but its style – the sober, detached recitation of news reports inherited from the manuscript newsletter – had little in common with that of the more engaged and discursive news pamphlets.

The newspaper, as it turned out, would have a difficult birth. Although it spread quickly, with newspapers founded in over twenty German towns in the next thirty years, other parts of Europe proved more resistant – Italy for instance was late to adopt this form of news publication. Many of the first newspapers struggled to make money, and swiftly closed.

The Devil's abroad in false Vellore

Always, one thing leads to another. I am researching obscure forms of rapid messaging in pre-telegraph environments. From that start, I end up, of course, in India at the time of the Great Mutiny in 1857 and the mystifying, and still not understood, appearance of bread loafs, chapatis, passed from village to village. It was a sign of something but that something was not understood by the British or the villagers at the time.

From the Great Mutiny, I end up with the predecessor Vellore Mutiny in 1806. Interesting in its own right.

From there, I reach an account of one of the survivors, Amelia Farrer, Lady Fancourt who wrote An Account Of the Mutiny at Vellore, by the Lady of Sir John Fancourt, the Commandant, who was killed there July 9th, 1806 in The Sydney Gazette and New South Wales Advertiser, 14 June 1842, p. 2. Incredible the lives and stories of just a couple of centuries ago.
At this moment, I gave up all for lost. I opened my dressing-room table drawer, and took out my husband's miniature, which I tied, and hid under my habit, and determined not to lose it but in death. I had secured his watch some time before, to ascertain the hour. I had hardly secured this much valued remembrance of my husband, before I heard a noise in the hall, adjoining my bed room. I moved softly to the door, and looking through the key-hole, discovered two sepoys knocking a chest of drawers to pieces. I was struck with horror, knowing their next visit would be to my apartments. My children, and the female servants were at the time lying on a mat, just before a door which opened into the back verandah, and which, at the commencement of the mutiny, seemed the safest place, – as shots were fired at the windows, we were obliged to remove as far as possible from them. I whispered to my Ayhal, that the sepoys were in the hall, and told her to move from the door. She took the children under the bed, and also begged of me to go there with them. I had no time to reply, before the door we had just left, was burst open. I got under the bed, and was no sooner there, than several shots were fired into the room; but, although the door was then open, no body entered. I took up a bullet which fell close upon me, under the bed. The children were screaming with terror, at the firing, and I expected our last hour was come, but willing to make one effort to save my babes, I got from my hiding place, fled into a small adjoining room, off the back stair-case.
And from there, I came across a poem about Vellore by one of my favorite poets, Sir Henry Newbolt. Very much a poet of the Empire, and a good one at that with many memorable poems and lines. Gillespie, his Vellore effort, is not, in my opinion, among the good ones but you can see the sentiment. Sir Rollo Gillespie was the commanding officer of the relieving force who rescued the few survivors. Another day, a few more nuggets of accidental knowledge.
by Sir Henry Newbolt

Riding at dawn, riding alone,
Gillespie left the town behind;
Before he turned by the Westward road
A horseman crossed him, staggering blind.

'The Devil's abroad in false Vellore,
The Devil that stabs by night,' he said,
'Women and children, rank and file,
Dying and dead, dying and dead.'

Without a word, without a groan,
Sudden and swift Gillespie turned,
The blood roared in his ears like fire,
Like fire the road beneath him burned.

He thundered back to Arcot gate,
He thundered up through Arcot town,
Before he thought a second thought
In the barrack yard he lighted down.

'Trumpeter, sound for the Light Dragoons,
Sound to saddle and spur,' he said;
'He that is ready may ride with me,
And he that can may ride ahead.'

Fierce and fain, fierce and fain,
Behind him went the troopers grim,
They rode as ride the Light Dragoons
But never a man could ride with him.

Their rowels ripped their horses' sides,
Their hearts were red with a deeper goad,
But ever alone before them all
Gillespie rode, Gillespie rode.

Alone he came to false Vellore,
The walls were lined, the gates were barred;
Alone he walked where the bullets bit,
And called above to the Sergeant's Guard.

'Sergeant, Sergeant, over the gate,
Where are your officers all?' he said;
Heavily came the Sergeant's voice,
'There are two living and forty dead.'

'A rope, a rope,' Gillespie cried :
They bound their belts to serve his need.
There was not a rebel behind the wall
But laid his barrel and drew his bead.

There was not a rebel among them all
But pulled his trigger and cursed his aim,
For lightly swung and rightly swung
Over the gate Gillespie came.

He dressed the line, he led the charge,
They swept the wall like a stream in spate,
And roaring over the roar they heard
The galloper guns that burst the gate.

Fierce and fain, fierce and fain,
The troopers rode the reeking flight:
The very stones remember still
The end of them that stab by night.

They've kept the tale a hundred years,
They'll keep the tale a hundred more:
Riding at dawn, riding alone,
Gillespie came to false Vellore.

Tuesday, July 26, 2016

From remarkably early in the age of the first printed books Europe’s rulers invested considerable effort in communicating with their citizens

From The Invention of News: How The World Came to Know About Itself by Andrew Pettegree.
This was not unproblematic, particularly for the traditional leaders of society who were used to news being part of a confidential service, provided by trusted agents. Naturally the elites sought to control this new commercial market, to ensure that the messages delivered by these news books would show them in a good light. Printers who wanted their shops to remain open were careful to report only the local prince’s victories and triumphs, not the battlefield reverses that undermined his reputation and authority. Those printers who co-operated willingly could rely on help in securing access to the right texts. Court poets and writers, often quite distinguished literary figures, found that they were obliged to undertake new and unfamiliar tasks, penning texts lauding their prince’s military prowess and excoriating his enemies. Many of these writings made their way into print. For all that this period is often presented as one of autocratic and unrepresentative government, we will discover that from remarkably early in the age of the first printed books Europe’s rulers invested considerable effort in putting their point of view, and explaining their policies, to their citizens. This too is an important part of the story of news.

The patriotic optimism of the news pamphlets served Europe’s rulers well in their first precocious efforts at the management of public opinion. But it posed difficulties for those whose decisions relied on an accurate flow of information. Merchants ready to consign their goods to the road had to have a more measured view of what they would find – news pamphlets that obscured the true state of affairs were no good to them if what was important was that their cargoes should safely reach their destination. The divisions within Europe brought about by the Reformation were a further complicating factor: the news vendors of Protestant and Catholic nations would increasingly reproduce only news that came from their side of the confessional divide. News therefore took on an increasingly sectarian character. All this led to distortions tending to obscure the true course of events. This might be good for morale, but for those in positions of influence who needed to have access to more dispassionate reporting the growth of this mass market in news print was largely a distraction. For this reason the rash of news pamphlets that flooded the market in the sixteenth century did not drive out the more exclusive manuscript services. The avvisi continued to find a market among those with the money to pay; in many parts of Europe confidential manuscript news services continued to prosper well into the second half of the eighteenth century.

Logical root causes are not necessarily the right root causes.

Well, that's thought provoking.

It is glib and sounds so easily right. But is it? Instinctively, I am very supportive of global trade. Strategically and in the long run, it absolutely increases the productivity and welfare of everyone. Short term it is, of course, disruptive and does destroy jobs in one location while creating more in another location depending on relative costs and productivity.

Trade hurts many people, skilled and unskilled. Hurts in the sense that it disrupts. Some people disrupted end up better off and some worse. On balance, over time, the benefits exceeds the costs but not for every person. There are people who definitely will suffer, even in the long run. Are the unskilled disproportionately affected in a negative way by global trade? I am not sure. Trade displaces lower value-add production and forces us nationally up the value add curve.

But those jobs lost to trade are not unskilled labor. They are, however, fungible skills easily relocated. Maybe it is splitting a hair to distinguish between unskilled and lower skilled.

I suspect the heart of the matter is really technology disruption which is a much clearer disruptor of jobs.

I am guessing that the linkage is that global competition (not trade per se) drives technology displacement. Technology displacement rewards the higher skilled and harms the lower skilled (and those that are less adaptive).

We end up with a pool of lower skilled people who also might be less adaptive (older, physically less capable, less optimistic or culturally resilient, etc.) who are being asked to let go of that which is familiar and make hard and expensive adjustments. Those are the ones who suffer.

Meanwhile, the younger, richer (buffers them from the hardships of adjustment), the brighter and/or better educated, benefit by being forced up the value curve earlier in their career when there are fewer sunk costs.

This dynamic would seem to explain in part the hollowing out of the middle class.

Even Kaus's observation about immigration seems subject to refinement. It is not that they are unskilled that is the distinguishing factor. It is that they are young. Disruption occurs and the young are more likely to adapt easier than the older.

I suspect that Kaus has captured correctly the popular formulation of the chain of causation. Hence the support of Bernie and Trump over the establishment candidates. The establishment protects itself and not citizens. The elite do not bear the real, direct cost of global competition, technology change and immigration.

The fact that the formulation is not quite right doesn't change people's conviction that it is right. It clearly sounds right.

The problem is that we need to know the real causation in order to understand what can really be done.

Even if we were to erect massive tariffs to restrict trade, that doesn't address the still existent competition. If everyone outside the country is reducing costs and becoming more efficient, then tariffs only delay the adjustment, not the fact that we will have to adjust.

Similarly, can technology development really be slowed, even if we wanted to? Again, likely that this is simply a systemic force that has to be accommodated and adjusted to.

That leaves immigration as the only lever likely to be subject to adjustment. Of course, even here, the latitude is probably a lot less than we would wish. Certainly we can do a lot better at reducing legal immigration (if we wanted to) and even illegal immigration. It is a matter of consistent and diligent enforcement. But as long as the immigration flow is demographically unbalanced (younger) then there will still be a disproportionate harm to the older, poorer and less adaptable.

I don't have an answer other than to use the regulatory processes to slow things down a little without actually opposing the adjustments.

I think that means that the issue is to change focus away from trade and technology as causal factors. Yes, immigration remains a focus for more effective management. Perhaps the focus most needs to be on helping citizens to adjust behaviors and values, make it easier and cheaper to acquire new skills, and think more creatively about how work can be designed to mitigate the impact of aging.

Which US state is closest to Africa?

Very interesting information. Virtually useless of course other than to remind us that the obvious is often not so obvious. From Which US state is closest to Africa? It's not Florida by Sean Kane.

I went with Cape Hatteras while Florida also seems a reasonable candidate. In fact, the answer is Quoddy Head Light in Maine. We are not talking about small differences either. Quoddy Head Light is closer to Africa than Florida by nearly nine hundred miles.

Click to enlarge.

Well somebody told us Wall Street fell

Another evening playing Spotify in the background. From Song of the South by Alabama

The line that caught my ear:
Well somebody told us Wall Street fell
But we were so poor that we couldn't tell
That resonates.

Monday, July 25, 2016

You've read the book so you better do it

George E.P. Box on how he became a statistician. From Wikipedia.
I want to tell you how I got to be a statistician. I was, of course, born in England and in 1939... when war broke out in September of that year, although I was close to getting a degree in Chemistry, I abandoned that and joined the Army. They put me in the Engineers (and when I see a bridge I still catch myself calculating where I would put the charges to blow it up).

Before I could actually do any of that I was moved to a highly secret experimental station in the south of England. At the time they were bombing London every night and our job was to help to find out what to do if, one night, they used poisonous gas.

Some of England's best scientists were there. There were a lot of experiments with small animals, I was a lab assistant making biochemical determinations, my boss was a professor of physiology dressed up as a colonel, and I was dressed up as a staff sergeant.

The results I was getting were very variable and I told my colonel that what we really needed was a statistician.
He said "we can't get one, what do you know about it?" I said "Nothing, I once tried to read a book about it by someone called R. A. Fisher but I didn't understand it". He said "You've read the book so you better do it", so I said, "Yes sir"