Know your data, show your data: A rant

I am finishing up my first year of doctoral level political science studies. During that time I have read a lot of articles – approximately 550. 11 courses. 5 articles a week on average. 10 weeks. 11×5×10=550. Two things have bothered me immensely when reading these pieces: (1) it’s unclear authors know their data well, regardless of it being original or secondary data and (2) the reader is rarely showed much about the data.

I take the stance that when you use a dataset you should know it well in and out. I do not just mean that you should just have an idea if its normally distributed or has outliers. I expect you to know who collected it. I expect you to know its limitations.

For example I have read public opinion data that sampled minority populations. Given that said populations are minorities they had to oversample in areas where said groups are over represented. The problem with this is that those who live near co-ethnics are different from those who live elsewhere. This restricts the external validity of results derived from the data, but I rarely see an acknowledgement of this.

Sometimes data is flawed but it’s the best we have. That’s fine. I’m not against using flawed data. I’m willing to buy most arguments if the underlying theory is well grounded. To be honest I view statistical work to be fluff most times. If I don’t really care about the statistics, why do I care if the authors know their data well? I do because it serves as a way for authors to signal that they thought about their work. It’s similar to why artists sometimes place a “bowl of only green m&ms” requirement on their performance contracts. Artists don’t know if their contracts were read, but if their candy bowl is filled with red twizzlers they know something is wrong. I can’t monitor whether the authors took care in their manuscripts, but NOT seeing the bowl of green only m&ms gives me a heads up that something is off.

Of those 500+ articles I have read only a handful had a devoted descriptive statistics section. The logic seems to be that editors are encouraging that stuff be placed in appendices to make articles more readable. I don’t buy that argument for descriptive statistics. Moving robustness checks or replications to the appendices is fine, but descriptive stats give me a chance to actually look at the data and feel less concerned that the results are driven by outliers. In my 2nd best world all dependent variables and major independent variables would be graphed. If the data was collected in differing geographies I would want the data mapped. In my 1st best world replication files with the full dataset and dofiles would be mandatory for all papers.

I don’t think I am asking too much here. Hell, I am not even fond of empirical work. My favorite academic is Peter Leeson (GMU Econ & Law) and he rarely (ever?) does empirical work. As long as empirical work is being done in the social sciences though I expect a certain standard. Otherwise all we’re doing is engaging in math masturbation.

Tldr; I don’t trust most empirical work out there. I’ll rant about excessive literature reviews next time.

Advertisements

From the Comments: The medieval Dark Ages were indeed dark

Dr Stocker answers my question about non-European canons of liberty:

Hello Brandon, sorry I didn’t have time to check the comments on this earlier. I don’t really want to say there was a 1000 year dark age for thought about liberty, but in terms of big recognised classics, it does look like a ‘Dark Age’.

Sadly I’m not equipped to discuss what was going on outside ‘Christendom’, the Medieval Christian world which largely corresponded with Europe particularly after the Arab (and in the west Berber) Muslim conquests in north Africa and south west Asia, so in what had been the Byzantine Empire outside its Balkan and Anatolian heartland.

I’m very slightly better qualified to discuss the Muslim world of this time than the cultures further east, and as far as I can see despite the riches of Muslim intellectual achievement, and the building of legal traditions, there is no major figure who could be described as pro-liberty though as with Aquinas, William of Ockam and other major political writers in Christendom of the time, there is an interest in law and respect for law from the sovereign power. I personally feel it’s a bit of a stretch to include that in any kind of liberty tradition, though the rule of law ideas to feed into it and to some degree pick up on antique republican thought, but largely in its empire of laws aspect rather than other aspects of political and social liberty.

There is a lot of really important and interesting stuff going on further east, particularly in China and India, going back to at least the time of Aristotle in Greece, in terms of philosophical, ethical, and political thought, and institutional innovation. On the institutional side though, I can’t see anything that looks very ‘republican’ or holding power accountable, or valuing challenges to excessive power. I’m sure there are texts that are important for liberty minded people to read, and some things some absolute rulers did like Buddhists who tried to abolish slavery, worth knowing about, but I’m just not competent right now to deal with this stuff properly. It is becoming better known in the west and that is going to produce results in the liberty community. I’ll see if I’m ever ready to engage, I’ve got some iras about how to get there from particular interests of mine, but it needs time.

In discussing Asian political traditions, one issue which is being discussed a lot is state hill communities in southern Asia, though from a collectivist anarchist position rather than an individualist anarchist position, the discussion has been picked up to some degree from an individualist point of view (Peter Leeson, the George Mason economist for example) and I think we’ll see more of that over time. That issue of hill peoples brings me onto something else.

Knowledge of the political structures of hill peoples comes from anthropologists (particularly the Yale anthropologist and agrarian studies specialist James C. Scott, a collectivist anarchist in inclination) rather than from texts in political theory by those stateless peoples. They were illiterate and maybe deliberately so to protect themselves from the low land state observations. Any political philosophy (or indeed philosophy of any kind) of such people comes from looking at the assumptions and everyday ‘ideologies’ of their lives. A big thing on that issue which has been getting increasing interest is that until the late 18th century European histories of philosophy included that kind of implicit philosophy of illiterate peoples observed in an ‘anthropological’ way by ancient historians like Tacitus and Herodotus. There has been a modern equivalent, roughly speaking, to that Herodotus/Tacitus observation of the supposed beliefs of peoples who seem very foreign, which is African philosophy, as studied by African scholars and outside Africa, largely by African-American scholars in US universities. This has engaged with an anthropological-philosophical study of the belief systems of colonised and pre-colonial African peoples.

There is a scholar known to me by personal acquaintance as well as academic reputation working on that sort of approach to non-literarate or not very literate non-urban societies round the world. That is Justin E.H. Smith a (white) American based at the University of Paris, who has a book due out on this in a few years, I’m certainly looking forward to it. That leads me to your point about ideology.

I agree that there is a valid area of study of philosophy, political throughout etc as it exists outside ‘ideology’ as written texts on those theme. It may have some relation to ‘ideology’ as everyday assumptions, though with less of the control/conformity associations of ‘ideology’. I am not on the whole the right person to say much about this, but over time I might be able to post a few things. I’m thinking of taking a step in that direction for next week’s post, which I’m thinking could be on the Medieval Iceland Eddas (heroic poetry) as it relates to a society, which apparently had very little in the way of a central state. That will mean breaking the timeline I’m working through, but that’s OK as I now realise I meant to cover the Roman historian Tacitus, but forgot, so next couple of posts will probably go back in history. Just working on a post on a 14th century English legal thinker, John Fortescue, for this weekend.

You can all read Dr Stocker’s promised Fortescue post here if you haven’t already (it’s excellent, of course). I have been interested in liberty from a non-European point of view ever since I first became interested in liberty (2008, thanks to Ron Paul’s presidential run, and I have always been interested in non-European cultures). A part of me wants to believe that there is an unwritten code of liberty to be found within all societies, and I think that there is a case to be made for this, if you look closely enough.

However, I was doing a search for liberal political parties throughout the world (liberal means libertarian!) and I was genuinely shocked at how few liberal parties there are outside of continental Europe and the Anglo-Saxon world. Even Latin America, long the West’s red-headed stepchild, has a dearth of liberal political parties.

Most parties in the non-European world are based around ethnicity, nationality, or socialism. The fact that socialism is ambiguous enough that it can allow for a narrative that incorporates ethnicity or nationality into its premise probably accounts for the popularity of these political parties. (So, for example, a political party that serves the interests of an ethnic group in a post-colonial state will often name itself the “National Party of Post-Colony,” or the “People’s Party of Post-Colony.”) This is still a disheartening trend, though. In the US, both major political parties are essentially liberal, and in continental Europe most of the political parties are liberal in fact if not in name.

I note here that factions and not parties are ultimately what drives drives politics, but the lack of liberal political parties can still us something about a society’s cultural mores.

For some reason this superficial political observation, coupled with Barry’s astute thoughts, reminds of this old post by Jacques on knowledge, language, and information.

Somalia and Anarchy: Links Edition

  1. I am too lazy to write much more on Somalia right now (you can always check out my latest piece again if you are really itching for something satisfying), so I have compiled a list of great pieces I have read over the past couple days on Somalia, Anarchy, and the idea that post-colonial states ought to fail more often than not.
  2. Jeffrey Herbst and Greg Mills argue over in Foreign Policy that the Congolese state needs to fail if the region is to ever know peace again.
  3. Over in the New York Times, Alex de Waal argues along the same lines that I have: that Somalia as it stands is a bad idea, and that much more decentralization is needed for it to effectively flourish.
  4. The Mises Institute has two wonderful articles (one by an anthropologist and one by a lawyer) on why anarchy has been great for Somalia, despite the government interventions imposed upon the Somalis by the West over the past two decades (and, really, much longer than that, but I digress).
  5. Political Economist Chris Blattman raises the flag of caution, though.  How do we really know that more states will be better for the people living in these regions?
  6. Co-editor Fred Foldvary defends anarchism’s good name after the (government-initiated) looting in Iraq.
  7. And last but not least, Cato Unbound, one of my favorite places to visit, had an excellent symposium on anarchism awhile back (like, 5 years ago).  Here is Pete Leeson’s lead essay, in which Somalia is specifically used to illustrate his points.  Be sure to read the responses of the other members in the exchange, too.

Have a great weekend, and have fun with all the reading!  One of the things that really bothers me is the example of Somalia that is thrown out in favor of government over liberty.  I really hate having to take the time to explain to people that the problems in Somalia are created by the government!  It’s like screaming at a brick wall…