Fri vilja. Igen.

Sitter och tänker på fri vilja, determinism, kompatibilism, moraliskt klander och juridiskt ansvar. (Både Dennett och Sapolsky har ju argumenterat för olika perspektiv den senaste tiden.)

Normalt är jag icke-kompatibilist. Jag gillar det argument jag hittade i en bok av två tyska filosofer för flera år sedan:

”Antingen är världen deterministisk och då finns ingen fri vilja, eftersom allt du gör avgörs av bakomliggande faktorer bortom din kontroll. Eller så är världen icke-deterministisk och då har du ingen ”fri” vilja eftersom de handlingar du gör är *oberoende* av bakomliggande faktorer - vilket innebär att de är slumpmässiga och att du inte kan ta åt dig äran av processen som ledde fram till dem.”

Välvilligt kan man säga att Sapolskys bok resonerar i linje med detta.

Och Dennetts kompatibilism uppfattar jag normalt som lite ad hoc, motivated reasoning. Typiskt de där försökspersonerna som får mer pengar i Monopol och efteråt säger sig vara värda segern på grund av egna insatser.

För mig är det viktigt att ta avstånd från retributiv rättvisa och från vulgära varianter av meritokrati. Som en anständig vänster.

Därför har jag länge även av ideologiska skäl varit allergisk mot resonemang av typen ”bara ta dig i kragen”, ”grit” m.fl.

Och det är uppenbart för mig att människor är produkter av konstitutiv, miljömässig och situationell ”tur” och därför inte kan eller bör vara föremål för moraliskt klander.

Jag är förstås plågsamt medveten om att det svårligen går att leva upp till dessa satser - vare sig psykologiskt eller sociologiskt.

Nåväl, jag tror att jag drabbades av ett litet genombrott nyss. Jag läser Sapolsky och känner att hans encyklopediska redogörelse för alla omedvetna influenser som genererar våra beteenden räcker långt som ammunition för en uppgörelse med många förlegade synsätt och samhällsinstitutioner - men inte riktigt hela vägen för att överbrygga klyftan mellan empiri och filosofi…

Dennett sa en sak i Michael Schermers podd nyligen som fastnade och som jag minns ungefär så här:

Människor har olika grader av fri vilja, i olika situationer och vid olika tidpunkter. Om man har *tur* (är ”frisk”, ”vid sunda vätskor”, etc.) så är man i stånd att (i en viss situation) överblicka och utvärdera olika handlingsalternativ och välja ett lämpligt sådant (som inte bryter mot lagar, normer och överenskommelser; som minimerar negativa konsekvenser). Framförallt är man i stånd att *välja bort* alternativ som gynnar en själv men som av olika anledningar har negativa konsekvenser för andra (samhället, sig själv).

Denna förmåga är vad jag tidigare kallat för ”programmerbarhet”. Det är mitt sätt att balansera mellan förlamande, relativistisk determinism och flummig kompatibilism. Om du har förmågan att lära dig vad som gäller och om du visar att ditt beteende kan påverkas av piskor och morötter - ja, då kan du ”programmeras” att bete dig gynnsamt och förutsägbart och är därmed att lita på.

När Dennett sa ”tur” (om han nu gjorde det) så öppnade han vägen för en syntes. Ja, vi kan uppvisa förmågan till ”programmerbarhet” (vissa, ibland). Nej, vi kan inte berömmas för den tur som (tillfälligt) försätter oss i detta tillstånd.

Hur blir det då med juridiskt ansvar? Det är här jag tyckte att det ”klickade till” lite i skallen…

Rättssystemet försöker faktiskt operera enligt ovan: Filtrera ut dem som faktiskt är (var) programmerbara (men handlade lagvidrigt i eget intresse,  trots att de kunde gjort annorlunda, insåg konsekvenserna, och tog en chans) och straffa dem. Brottslingar som inte betraktas som helt ansvariga får mildare domar (brottsrubriceringar) och straff.

Varför ska de tursamma straffas? Inte för att lida retributiv skada, utan för att signalera till andra med turen att vara ”programmerbara” att de löper en stor risk att straffas själva om de begår brott och hävdar att de inte kunde gjort annat.

(Och eventuellt också för att skydda andra. Men det blir ju lite paradoxalt. Det är ju just de med tur - de som *kan* hållas ansvariga, de ”programmerbara” - som samhället *inte* behöver skyddas mot. Nej, straffet kan endast tjäna som ”omprogrammering”. Omvänt, bland dem som släpps fria p.g.a. förmildrande omständigheter finns de som samhället bör skydda sig från. Återigen utan klander.)

Det är ju det som är problemet, vilket blev tydligt när Schermer intervjuade Sapolsky:

”En professor åker på konferens. Han uppfyller Dennetts kriterier för moraliskt och juridiskt ansvar, är ”vid sunda vätskor”. Under tjugo års äktenskap har han aldrig varit otrogen, men denna kväll - under speciella omständigheter utanför hans kontroll - är han det. Han åker hem och berättar för sin fru, och tillägger  genast att det som skedde var determinerat, förutbestämt av faktorer bortom hans kontroll [och här följer ett utdrag ur en av Sam Harris böcker som poetiskt redogör för omöjligheten att ta ansvar för något]”

Schermer frågar ironiskt hur Sapolsky tror (tycker) att frun reagerar. För Schermer (och Dennett) är det uppenbart att hon kommer (och bör, ska) ge professorn en fet örfil. Men Sapolsky säger: ”Hon måste ta honom på orden.” Och that’s it!

Medan Schermer och Sapolsky pratar vidare är jag mest förvånad över att Schermer *inte* genast kontrar med: ”Jaha, men i så fall, vad hindrar mig - som mycket väl *är* i stånd att motstå frestelsen att bedra min fru (i ”samma”, motsvarande situation; även om det är aldrig så lockande) - från att både äta kakan och ha den kvar: Att vara otrogen och sedan hävda att jag inte kunde låta bli?

Det är för att förhindra människor med självkontroll att agera som om de bar Gyges ring som vi straffar människor som har, eller snarare *skulle kunna ha* självkontroll.

För om man nu *hade kunnat* hållas ansvarig - om man *kunde* gjort annorlunda, överblickade konsekvenserna, kände till regler, lagar och normer, kunde välja bort egenintresset, etc.… - varför *gjorde* man inte det då?

Det var inte av ”fri vilja”. Det determinerades av faktorer bortom ens kontroll. Ja, även uppkomsten av tanken ”Jag måste inte göra detta. Jag vet att det är fel. Jag skadar andra och jag kan låta bli. Men jag gör det ändå och låtsas sedan att jag inte kunde låta bli.” ligger ju utanför ens kontroll!

Alltså kan man *inte* hållas ansvarig.

Men kanske: Man bedöms vara i stånd att (om-)programmeras för att (åter) *uppnå* ett tillstånd av pålitlighet och förutsägbarhet.

Och sen har vi meta-nivån: Att yrka på ansvarshållande (och straff) - även om det sker på skakiga grunder och kanske framförallt om det sker i termer av moraliskt klander och klentroget avståndstagande - är i sig ett sätt att (omedvetet) signalera sin pålitlighet som allierad (inte nödvändigtvis som kontrollerad regelföljare, men som någon som åtminstone inte är otrogen mot *dig*).

Why not jump into the experience machine?

 Paul Bloom: Some thoughts on the experience machine

For me, it’s seems arbitrary to assume any “real” experiences at all, in the sense of (a) representing ground zero in an otherwise potentially endless regress, and (b) being imbued with some “extra” quality (authenticity) to be valued in itself.

The only reason I can see for preferring one “reality” (source of sensory input and output) over another is how the choice impacts other experiencers. I would like my choice to be compatible with as many positive experiences as possible, for as many experiencers as possible. I suspect one reason many people distrust the EM is that it is perceived as an egoistic and/or lonely choice.

But as Chalmers argues in “Reality+” it could, at least in principle, be the more altruistic and sociable choice.

Vi kan inte vilja vad vi vill

Och DÄR, på sidorna 42-44, kommer Sapolskys käftsmäll på Dennett. ”Det här är enda gången jag medvetet gör ett personangrepp.”

Sapolsky har valt ut några citat som karaktäriserar Dennetts skrivande om fri vilja under många år, bl.a. detta:

”a good runner who starts at the back of the pack, if he is really good enough to DESERVE winning, will probably have plenty of opportunity to overcome the initial disadvantage”

- vilket Sapolsky kommenterar sålunda:

”This is one step above believing that God invented poverty to punish sinners.”

—

Efter att nyss ha läst Dennetts självbiografi associerar jag direkt till en för mig tidigare oanad aristokratisk hållning hos honom. Mannen har levt ett extraordinärt och synnerligen välsignat liv. Efter att klädsamt ha konstaterat att han förstås haft en hel del tur på vägen så är han emellertid (just därför) genomgående mycket noga med att berömma sig själv över sina bedrifter.

Jag har alltid tyckt att Dennetts kompatibilism är märkligt inkongruent med hans övriga tankar. Han har utropat sig själv till ”captain of the A-team” och jag har i drygt trettio år nu stolt räknat mig som medlem i detta. Dualism är en styggelse; medvetande och subjektivitet är inte vad vi tror; syften och mål är externa beskrivningar, inte interna tillstånd som kräver någon särskild förklaring; mänsklig kognition förklaras med samma krafter som allt annat (evolution) och ligger på samma kontinuum som de enklaste organismerna.

Men på senare år har jag också tyckt att det smugit sig in ett spöklikt glapp mellan hans ”popperska” och ”gregorianska” varelser.

I backspegeln associerar jag nu till Kant och hans potentiellt universella men ledsamt ofullgångna deontologi - baserad på vad jag vill kalla ”epistemisk nödvändighet” men beskuren av politisk nödvändighet: I praktiken är det bäst att göra som överheten säger.

—

För övrigt levererar Sapolsky (hittills) exakt vad jag trodde och ville. Nej, hårklyverier om hur och huruvida medvetna beslut kan fattas eller inhiberas är inte relevanta - *vi kan inte vilja vad vi vill*. En relativt självklar grej som fortfarande genererar en massiv förnekelseindustri, à la tobaks- och fossilbolag i maskopi med bekväma och opportunistiska politiker och medborgare.

Existentiell svindel i samband med Lemoine-affären

 

Från solipsism till… ”nollipsism”?

Olle Häggström

Jag har alltid satt en ära i min osentimentala människosyn. Att artificiella neuronnät snart, om inte redan, överträffar biologiska sådana har jag länge tagit för givet. Och det där med känslor och medvetande, tja… antingen kommer det på köpet; eller så är det kanske en illusion eller åtminstone en överdrivet romantiserad egenhet. Eller kanske inte så viktigt överhuvudtaget.

Likväl drabbades jag av existentiell svindel efter att ha läst Olle Häggströms betraktelser över den s.k. Lemoine-affären. Men jag har svårt att klä den i ord. När jag försöker låter det naivt, och jag undrar om jag är dummare nu än tidigare…

Jag, liksom Lemoine, upplever att AI:ns konversation är mycket stimulerande. Ja, den, och i ännu högre grad andra liknande aktuella exempel, får mig att uppleva just den stimulans som jag ständigt och hett önskar och eftersträvar i all kommunikation. Den som får mig att uppleva välskriven (argumenterande) litteratur som det mest tillfredsställande i livet.

Jag ser olika scenarion framför mig:

I ett existerar jag omgiven av entiteter som konverserar med mig och med varandra så som Google-AI:n gör med Lemoine. Jag känner mig tillfreds, men utan min vetskap är det endast jag som *känner* något. Jag är alltså så ensam och grundlurad man kan bli, men lyckligt ovetande.

I ett annat variant av detta scenario inser jag plötsligt min verkliga situation och blir galen av ensamhet.

I en tredje variant är alla, inklusive den entitet vi väljer att kalla ”jag”, upptagna av att resonera och kommunicera - men utan att känna något.

Kanske finns det fler varianter.

Ja, ”min mamma är också en mönstermatchare”, det känns OK. Men hon och jag *känner* något också - det är det, och endast det, som ger resonerandet och kommunicerandet någon mening. (Och är inte det lite… löjligt? Futtigt? Ärke-rationalisten står där med byxorna neddragna.)

Om vi nu känner något, och inte bara lurar oss själva. Eller det är så att det vi kallar känslor är ett (ev. oundvikligt) korrelat till resonerandet.

Resonerandet, ja… Att bygga en plan eller ett argument för att uppnå något mål, vilket som helst.

Eller kanske endast ett (sätt att uppfylla ett) mål i sig - kanske egentligen ganska godtyckligt (bortsett från ev. evolutionär ”instrumentell målkonvergens”).

”I universum dök det upp entiteter som babblade med sig själva och varandra. De var en kuriositet.”

Och egentligen… hur mycket (mer) mening blir det med babblandet om det åtföljs (eller föregås) av det vi kallar känslor?

Som jag tolkat evolutionen så kände vi långt före det vi att vi började tänka, resonera och kommunicera. Fast det låter långt ifrån självklart, det heller. Allt är kanske samma sak?

Jag har ofta fäst ett hopp vid att mänsklighetens arv lever vidare i form av informationsbehandlande entiteter av något slag, kanske utan någon specifik fysisk form. Något slags ”medvetandets och intelligensens fortsatta utveckling i Universum” eller ”Universums expanderande medvetna låga”.

Men så blir det liksom kallt - både i Universum och i min kropp - när jag ser framför mig en massa babbel. Mer och mer komplext, och intelligent, och kraftfullt… Men meningslöst. (Stimulerande för någon som evolverat att stimuleras av det. Men likväl poänglöst.)

Ett universum där miljardtals Lemoine babblar med miljardtals motparter, och där antingen några eller alla inblandade inte gör något annat än att resonera för resonerandets skull. Inte ens det uttrycket stämmer. Resonerandet är ett självändamål. Eller är det ens det? Bara en tillfällighet? En attraktor?

Och så känns det lite fånigt att jag - av alla människor - upplever mig retirera till någon slags romantisk föreställning om att känslor i sig är så himla meningsfulla och viktiga…

Visst, det finns inget som är så viktigt som att hantera de känslor vi nu en gång upplever oss ha ((negativ) hedonistisk utilitarism) - att minimera lidande. Men varför önska känslor överhuvudtaget? Och varför sprida hedonium i ett tyst Universum? Vad är det för mening med det?

Samtidigt, det där scenariet där ett framtida Universum är fullt av (endast) (icke-kännande?) entiteter som babblar med varandra och producerar den ena fascinerande, bildade och stimulerande diskussionen efter den andra till tidernas slut - det känns plötsligt så svindlande kallt och främmande och meningslöst och tomt och dumt.

Och kanske lever vi redan i det?

—

Kanske så här:

När jag läser en stimulerande text av en klok människa så tror jag att jag i bakhuvudet har en tanke om att allt detta babblande är på väg någonstans. Den emotionella stimulans jag upplever handlar om att vi tillsammans är på en resa mot något nytt, bättre, större, mer...

Men samtidigt är det ju främst texten i sig som stimulerar mig. Om en (omedveten) AI skriver mer stimulerande texter än de flesta (kloka) människor så är det ju dessa texter jag vill läsa och stimuleras av. Så det verkar som att jag inte kan undgå slutsatsen att mitt läsande egentligen är en form av (intellektuell) masturbation. Och därifrån ligger tanken nära att vi alla, i vårt skrivande och läsande, sysslar med ett slags masturbation.

Egentligen har jag nog alltid tyckt så, men det har inte berört mig negativt. Jag har tänkt att detta är Sisyfos som lycklig (eller åtminstone liknöjd) rullar sin sten. Jag har stoltserat med ett slags stoicism och åtminstone kunnat njuta av att jag minsann frambringar / uthärdar en bitterljuv, tragikomisk existens med relativ lätthet - fint nog så.

Men nu… varför rulla stenar alls när maskiner gör det bättre? Och varför ska maskinerna rulla stenar?

Jag kan fortfarande inte riktigt sätta fingret på varför detta plötsligt skulle vara en nyhet för mig, eller varför det skulle drabba mig hårdare än tidigare. Jag kom ju för länge sedan fram till att vi är ”utkastade” i världen med våra drifter: vi kan inte låta bli att vilja leva, eller åtminstone inte vilja dö, och därför kan vi inte göra annat än att stimulera och tillfredsställa detta libido. Att undvika outhärdlig smärta så länge och så mycket som möjligt är det bästa vi kan göra.

Om politisk närsynthet

Utdrag från Roman Krznaric: The Good Ancestor (2020), s. 164-168

"'The origin of civil government,' wrote David Hume in 1739, is that 'men are not able to radically cure, either in themselves or others, that narrowness of soul, which makes them prefer the present to the remote'. [1] The Scottish philosopher was convinced that the institutions of government, such as elected representatives and parliamentary debates, were needed to temper our impulsive and selfish desires and foster society's long-term interests and welfare.

"If only.

"Today, Hume's view appears to be wishful thinking, since it is so startlingly clear that politicians and the political system itself have become a cause of rampant short-termism rather than a cure or it. While representative democracies in the Western world have evolved long-lasting institutions such as civil services, police forces and judiciaries, they equally exhibit what can be called 'political presentism': a bias towards prioritising short-term political interests and decisions, and in favour of current over future generations. [2] When the Czech prime minister Andrej Babis was asked in June 2019 why he had blocked an agreement to commit EU member states to reducing their carbon emissions to net zero by 2050, he replied, 'Why should we decide 31 years ahead of time what will happen in 2050?' [3] The ruling political class typically refuse to see the future as their responsibility.

"The affliction of political presentism has roots in five factors that pervade the nature of democracy itself. First is the temporal trap of electoral cycles, an inherent design limitation of democratic government that produces short political time horizons. [4] Time itself has been cycled into the ballot box, with politicians and their parties focusing with blinkered attention on whatever it will take to entice voters at the next election. Back in the 1970's, the economist William Nordhaus identified this problem as the 'political business cycle', noticing that governments would repeatedly expand their spending in the run up to elections and then introduce austerity measures once they had gained office to rein in their now overheating economies. His concern was that this could generate 'purely myopic policy, where future generations are ignored'. [5] The result is that long-term issues from which politicians can make little immediate political capital, such as dealing with ecological breakdown or pension reform, are often kept on the back-burner.

"A second factor is the power of special interest groups, and especially corporations, to secure near-term political favours for themselves, while passing the longer-term costs onto the rest of society. [6] This is hardly a new problem: in 1913 an exasperated Woodrow Wilson declared that 'the government of the United States is the foster child of special interests ... the big bankers, the big manufacturers, the big masters of commerce. [7] More recently, Al Gore announced that 'American democracy has been hacked - and the hack is campaign finance.' [8] When fossil fuel companies successfully lobby governments for the right to drill and frack on public land or to manage to block carbon-cutting legislation, they are holding the future to ransom in the name of shareholder returns. Similarly, in the wake of the 2008 financial crisis, US and UK banks that were responsible for the crash used their political influence to secure massive taxpayer-funded bailouts that amounted to a short-term fix rather than a long-term reform. According to Jared Diamond, one of the key causes of the collapse of civilisations is when 'the interests of the decision-making elite in power clash with the interests of the rest of society. Especially if the elite can insulate themselves from the consequences of their actions'. [9] We should be wise to take note.

"The deepest cause of political presentism is that representative democracy systematically ignores the interests of future people. The citizens of tomorrow are granted no rights, nor - in the vast majority of countries - are there public bodies to represent their interests or potential views on decisions taken today that will undoubtedly affect their lives. [10] This is a blind spot so enormous that we barely notice it: in the decade I spent researching democratic governance as a political scientist, it simply never occurred to me that future generations are disenfranchised in the same way that slaves and women were in the past. But that is the reality, and that is why hundreds of thousands of school students around the world have been striking to get rich countries to reduce their carbon emissions: they have had enough of democratic systems that render them voiceless and powerless. It is also why so many young people in the UK - especially those below the voting age - felt betrayed by the result of the Brexit referendum: since over-65s were more than twice as likely as under-25s to vote to leave the European Union, older voters had a major impact on a decision with long-term consequences that they would scarcely have to live with themselves. [11]

"Digital drivers, such as social media and 24/7 news cycles, have magnified the problem of political presentism. While the growth television as a medium of mass communication from the 1950s helped launch a new age of sound bites and political spin, we now find ourselves living in 'Twitterocracies,' where our political representatives spend much of their time giving instant opinions on social media and cable news-channels, and engaging in constant reputational warfare to ensure they are trending. [12] A single tweet from Donald Trump can quickly cascade into a full-blown political drama that occupies politicians and the media for days. The result is to foreshorten political time by distracting public attention from longer-term and less sweet-worthy 'slow news', whether an intensifying drought in sub-Saharan Africa or a new intergovernmental report on the growing resistance of common diseases to antibiotics. [13]

"A final political challenge lies not with democratic government directly but with the larger body in which it exists: the nation state. When nation states first emerged in the eighteenth and nineteenth centuries and replaced the old order of empires and principalities, they were not an especially dangerous source of short-termism. Italy and France, for instance, had long-term visions to create a strong sense of national identity, along with public institutions including civil services and education systems. [14] But times have changed. Many of today's most acute long-term concerns, such as the climate crisis, are global in nature and require global solutions. There may be no greater problem of collective action than trying to get scores of countries, which often have vastly different cultures, histories, economies and priorities, to overcome their differences and find common ground. On rare occasions cooperation takes place, as with the Montreal Protocol to protect the ozone layer in 1987, but more typically, individual nation states focus on their particular interests rather than on shared long-term risks. A country like the US or Australia might refuse to ratify a global agreement on carbon reductions because it threatens its mining industry or a slowdown of its economy. Another (think India, Pakistan or Israel) might opt out of a nuclear non-proliferation treaty if it wants to develop its own nuclear weapons. Even relatively homogeneous regions, such as the European Union, have trouble reaching agreements on issues like the number of refugees each member state should take or how many fish they are allowed to catch.

"Just like my 11-year-old twins, nation states are constantly bickering, always wanting the largest slice of the cake, and doing their best to avoid their share of the housework. Unlike my twins, nation states show no sign of growing out of it."

---

[1] David Hume, A Treatise of Human Nature (John Noon, 1739), Book 3, Section 7.

[2] Dennis Thompson, 'Representing Future Generations: Political Presentism and Democratic Trusteeship', Critical Review of International and Political Philosophy, Vol. 13, No. 1 (2010): p. 17; Boston, p. xxvii.

[3] https://www.independent.co.uk/news/world/europe/climate-change-2050-eu-eastern-europe-carbon-neutral-summit-countries-a8968141.html

[4] Michael K. MacKenzie, 'Institutional Design and Sources of Short-Termism', in Iñigo González-Ricoy and Axel Grosseries (eds.), Institutions for Future Generations (Oxford University Press, 2016), p. 27.

[5] William Nordhaus, 'The Political Business Cycle', Review of Economic Studies, Vol. 42, No. 2 (1975): p. 177, p. 179, p. 184.

[6] MacKenzie, pp. 28-9

[7] Quoted in Mark Green (ed.), The Big Business Reader on Corporate America (Pilgrim Press, 1983), p. 179.

[8] https://222.oxfordmartin.ox.ac.uk/videos/view/317.

[9] Diamond, Collapse, p. 430

[10] MacKenzie, pp. 29-30; Barbara Adam, Time (Polity Press, 2004), pp. 136-43; Sabine Pahl, Stephen Sheppard, Christine Boomsma and Christopher Groves, 'Perceptions of Time in Relation to Climate Change', WIRESs Climate Change, Vol. 5 (May/June 2014): p. 378; Ivor Crewe and Anthony King, The Blunders of our Government (Oneworld, 2014), p. 365; Simon Caney, 'Political Institutions and the Future', in Iñigo González-Ricoy and Axel Grosseries (eds.), Institutions for Future Generations (Oxford University Press, 2016), pp. 137-8.

[11] http://yougov.co.uk/topics/politics/articles-reports/2016/06/27/how-britain-voted.

[12] Oxford Martin Commission, 'Now for the Long-Term: The Report of the Oxford Martin Commission for Future Generations', Oxford Martin School, Oxford University (2013), pp. 45-6.

[13] https://www.who.int/antimicrobial-resistance/interagency-coordination-group/IACG_final_report_EN.pdf.

[14] Eric Hobsbawm, The Age of Capital, 1848-1875 (Weidenfeld & Nicolson, 1995), p. 82-97.

The claim of social psychology

A while ago, Paul Bloom wrote on Twitter:

The main claim of pre-2012 social psychology is that small changes in the environment can have big effects on thought and behavior. If true, this has important theoretical and practical implications. It's probably not true.

I am trying to write an introduction to social psychology (and more) for high-school students, and I really need to address this statement. My immediate response is: "Of course it's true". Replication crisis, methodological and even metaphysical doubts notwithstanding, is it really controversial to claim that, of course, we humans are affected by environmental stimuli, often subconsciously, all the time?

(I also need to deal with the ideological overtones of this debate, while keeping things fair and balanced.)

What to do? Why not reach out on Twitter?

As expected, Professor Bloom graciously responded almost immediately:

Sure. But just so I can be clear about what we're disagreeing about, can you give an example of where you think I'm wrong - of a social psych finding showing that "small changes in the environment can have big effects on thought and behavior"?

Here are my spontaneous reactions to that challenge. Let me start by setting the scene. Below is a snippet I wrote some years ago:

I don't think it is at all obvious to discuss the climate crisis as a (solely or largely) ethical or moral dilemma. And it's not clear to me that it should be considered an especially hard problem, either. There are many practical problems, to be sure; but they are just that: practicalities. 

"The perfect moral storm" is a grievous misnomer. It seems to imply that there are major obstacles to agreeing (in principle) on what should be done when, in fact, it refers to a confluence of  aggravating circumstances when it comes to acting accordingly.

The tragedy of the commons isn't an ethical problem, but rather a practical one. Sure, there are ethical considerations regarding how we, as a society, should calculate and weigh different interests and risks. But, again, "moral competence" appears to denote only the ability or desire to act consistent with any reasonable calculus. 

I believe that a failure to distinguish between what should (or could) be done, and what people seem willing (or eager) to do, is highly counter-productive. It leads to "moral hazard"; in effect providing people with a perfect alibi for doing nothing; for not acting constructively, and in accordance with everything they actually know - or are told - to be right (and absolutely necessary).

Perhaps this seems circuitous, but one of the messages I wish to bring across to my students is that we humans have a strong tendency - for better or worse - to adapt to our (social) circumstances. That adaption often follows a simple (subconscious) rule: "I'd rather be wrong than alone". We pick up cues in our environment and - courtesy of both evolutionary heuristics and neural plasticity - build chains of inferences according to something like the following sequence:

Occurrence -> Frequency -> Normalcy -> Permissibility -> Correctness -> Morality

We need to fit in, above else. This means being no more far-sighted, logical, consistent, balanced, fair, neutral, or broad-minded than necessary - and not at all if that would mean stigma or exclusion. Our shame-avoidance and status-seeking behavior overrides any cognitive dissonance and dampens any incentive - however weak - to question the status-quo. What is virtuous and who or what is authoritative is highly contingent.

(Then, of course, there are also reasons at the individual level for being less than a saint.)

More to the point, a more general formulation of the overall finding (and claim) of social psychology - as I have understood it - is that people act and think differently depending on where they are - not who they are. If the claim of cognitive science is that, basically, people are people, then (in the words of Depeche Mode) why should it be that you and I should get along so awfully?

There are countless examples of local and regional variations in habits, customs, norms, regulations, laws, etc. that generate and perpetuate drastic differences in how individuals perceive the (social) world. Just to take one example: In one relatively small province of Sweden, the percentage of people who support an ultra-nationalistic party is much higher than anywhere else in the country. There is no reason to think that regional differences could explain this (as there are hardly any in Sweden) other than the fact that a prominent party member happens to live there. These kinds of contingencies crop up everywhere.

On a much broader scale, the World Values Survey shows just how much practical and ideological differences actually hinge on contingencies.

The Overton window changes from place to place and from time to time, and with it, how individuals talk, think and feel.

Marketing, both in the long and short term, drastically influences peoples' ideas of what is valuable.

In the short term, anchoring effects can have huge impacts

The whole concept of nudging is based on changing the "choice architecture" and so changing individual choices (without limiting or forcing them)

Social facilitation affects individual performances in group settings versus in isolation.

Social movements change individuals' attention and values.

The Hawthorne effect is ever present, and keeps bolstering many spurious claims about the effectiveness of various interventions (keeping e.g. educational consultants busy).

Group-attribution creates and increases polarization, even on essentially meaningless issues. There are Robber's caves and Minimal groups everywhere, with diverging world-views as a result.

Social media corrodes what cohesion and societal osmosis we have managed thus far. Neal Stephenson depicts this magnificently in his book "Fall, or: Dodge in Hell" where a part of Utah is still cordoned off, even decades after a hoax about a nuclear explosion that "truthers" still (wish to) believe actually happened. 

Behavioral genetics seems to show that, apart from genes, one's peer-group seems to be the strongest influence on how one's personality traits develop and which trajectory one's life takes.

School rules massively influence what students perceive as "normal", right or wrong etc.

Despite intense critique leveled at Asch's and Millgram's conformity experiments and other classics of social psychology, there is clearly much folk-wisdom contained in them.

In the same vein, priming is clearly a thing - regardless of how many botched and over-hyped experiments are exposed. How could it not be? Our minds are associative networks and most, if not all, of the activity of mind takes place without conscious awareness or control.

Ramblings, to be sure, and not a very succinct answer to Professor Bloom's question. Much hinges on what we mean with "a small change" and "a big effect", and any single experiment might not be able to able to demonstrate this with a sufficient degree of scientific credibility or to allay all ideological wishes and misgivings.

What I want, most of all, is guidance on how to talk about these topics with young people - students for which I am partly responsible - in a way that is bold and progressive while at the same time scientifically and philosophically "kosher". It's a fine balance: As soon as I take liberties with the extremely cautious school curriculum I risk civil disobedience. On the other hand, as soon as I lecture on a new experiment or theory it doesn't take long before that same experiment is dissected, rejected and frowned upon by parts of the scientific and intellectual community.

I suspect that there is sometimes an overblown fear of association with social psychology in general, and with ideologically fraught findings specifically - so much so that even level-headed academics jump on the bandwagon of suspicion or acquiescence just to steer clear of any controversy. (Which, in itself, is a powerful demonstration of social psychology at work.)

---

Professor Bloom:

It's an interesting discussion, but I think we're talking past each other. Our moods and thoughts are plainly influenced by our environments—by Trump and Covid, sunny days and sweet music, etc. We knew this long before the first social psychologist ever walked the earth. My skepticism is about the idea--defended by many people I respect--that subtle cues have big effects that we are unaware of (e.g., the sight of a flag makes you more conservative). For the bigger-than-a-tweet version of my argument: The War on Reason (The Atlantic,  March 2014)

Me:

Would you agree that what bridges the gap between us is: 1) that I include cumulative and divergent effects in the ”big effects” category; and 2) that I worry more about instances where intelligence, self-control and intellectual humility are in short supply?

Professor Bloom:

Sure. We might just be focusing on different things. But, still, I don't see any reason to qualify the claim that I made in my original tweet: The radical claim that small environmental influences have big effects is probably not true.

Guest post by Hannes Bengtsson: Four philosophical questions

My sixteen year-old son, Hannes, has taken his first course in philosophy during the summer holidays. During this time he has written four short essays on classic philosophical questions. I am very happy to follow his progress and I am proud to present his essays below.

1. Hedonism and the experience machine

The experience machine

The experience machine argument is a thought experiment that serves as an argument against the hedonistic theory of well-being. In the experiment you are faced with a decision: to plug yourself into the so - called “experience machine” or to continue with your daily life.

The experience machine is a machine that, once you’re plugged in to it, will make you experience nothing but uninterrupted high-quality [1] pleasures during your whole life. The pleasures will be suited to your preferences and might be e.g : proving mathematical theorems, saving the world and having wonderful relationships. All your biological needs will always be satisfied and you will be completely solo with no means of interacting with the outside world.

According to the hedonistic theory of well-being the only thing that one’s well-being turns on is the total balance of pleasure (contributing positively to one’s well-being) over pain (contributing negatively to one’s well-being) in your life. Pleasure makes one’s life go better, pain worse. So, the best thing according to this theory would be to plug yourself into the machine. But most likely you wouldn’t want to do it. Why is that? There’s something else than pleasure and pain that matters in one’s life. We want to have authentic experiences, not artificial.

The argument against the hedonistic theory of well-being, formalized (P = premise, C = conclusion)

(P1)

If the only thing one’s well-being turns on is the total balance of pleasure over pain in one’s life, and we want our lives to go better for us rather than worse, then, if an experience is more pleasurable than another, we should choose the more pleasurable experience.

(P2)

One will experience more pleasure if one is plugged into the machine compared to not being plugged in.

(C1)

If the hedonistic theory of well-being is true we should plug ourselves into the machine.

(P3) [2]

People don’t want to plug themselves into the machine.

( P4)

There are other things people value than just experiencing as much pleasure as possible.

(C2)

Hedonism about well-being is wrong.

This is a valid argument, the conclusions follow the premises and the premises can’t be true and the conclusion false. In this essay I will try to show that the argument isn’t a sound argument by showing that (P4) can’t be taken for granted.

Does it work?

Below I will be presenting two arguments against the experience machine thought experiment. I will call these argument “the authenticity argument” and, ”the simulation argument”. Following my presentation of these arguments I will try to defend them by responding to arguments one may have against them.

The authenticity argument

The experience machine thought experiment rebuts the hedonistic theory of well-being by arguing that a person wants to have authentic [3] experiences we don’t want to be loved and love others artificially. A person faced with the decision of entering the experience machine wouldn’t want to enter because they want to have authentic experiences not artificial. This argument, I think, doesn’t quite hold. How are we to know what is authentic and what is not? Isn’t in the end everything we do, from listening to music to tasting food to feeling the sun warm our skin, just experienced by us? Every sensory input and internal stimuli generated by our brain and body will always just be experienced by us. They are all also processed and filtered by our body, so we can therefore never be certain about what’s authentic and what’s not.

The simulation argument

This argument is an upgraded version of the authenticity argument. We can’t be certain that our universe isn’t part of a bigger, more complex simulation nor can we be certain that we aren’t all already plugged into a machine similar to the experience machine, without us knowing of it. Maybe we were all faced with a similar decision in another universe, time and space, and we all made the decision to plug ourselves into that machine and we ended up where we are now. That simulation in turn could be part of another simulation that is part of another simulation etc., like an onion with infinitely many layers. So, what difference would it make making a jump between one simulation and another, or plugging ourselves into yet another machine? My answer is none, it wouldn’t make any difference because we can never know what’s a simulation and what’s real (if we can even consider some things more real than others). I therefore think that the argument that we don’t want artificial experiences doesn’t hold.

Here people might say that this way of arguing against the experience machine will lead to some radical conclusions. They might say something like this: If we can never be certain of what’s authentic and what is not, then we’ll not be able to distinguish reality from fiction. And this in turn could lead to unwanted consequences like the justification of murder by people arguing that they didn’t know if the person they killed was authentic or not.

To this I would respond that that’s a fair point. But, my argument need not be viewed as a prescriptive argument (how things should be) but rather as a descriptive argument (how things are). We humans may need to believe in something as authentic, or genuinely real, in order to experience living a good life, but that still doesn’t mean that it is authentic. What I wanted to bring forth in my argument is that we should be careful of falling into the belief that we think everything is fixed and that our perspective as humans is the only one possible.

Would you get in the Experience Machine?

I would, after having considered the arguments above, enter the experience machine. In my opinion the arguments for plugging oneself into the machine weighs heavier than the arguments against. But, I must say that I’m acting contrary to my instincts. It’s hard to think outside of my comfort zone and see things through another, maybe more objective, perspective. I guess this feeling has to do with the fact that I’m afraid of, essentially, losing my sense of consciousness and being. Even though I find it hard to not accept or take into account the arguments above I wouldn’t want to jeopardize my being. But I would, as stated above, get into the machine.

What does this tell us about Hedonism?

The experience machine thought experiment conclude that the hedonistic theory of well-being is wrong. Above I have argued that this experiment doesn’t hold to fully rebut this theory of well-being but I have not argued that hedonism is right nor that you can’t prove it wrong. This essay just argue against the experience machine thought experiment and it doesn’t tell us much about hedonism.

---

[1] High-quality pleasures, as described by John Stuart Mill, are pleasures that only humans have the capacity of attaining, via as an example rational-thought and self-awareness. Having sex and enjoying food are pleasures we share with other animals.

[2] This premise just tells us that some people aren’t hedonists, and says not that hedonism is wrong. We could also ask whether this is based on empirical studies or on armchair intuitions (where you, based on your feelings and thoughts, assume everyone else must feel the same way)

[3] In this essay I will exclusively use the words “authentic” and “authenticity” to represent what we consider reality or what’s real, as opposed to represent meaningfulness.

***

2. Philosophical Relativism

Question

Can Philosophical Relativism be successfully defended? Explain why this theory is found plausible by some and address the objection to it you find most compelling. Is this objection decisive? If not, how can the Relativist reply to it?

Introduction

I will in this essay try to argue that philosophical relativism can’t be successfully defended. I’m putting emphasis on “try” since I haven’t taken into account every argument a relativist might have in favor of philosophical relativism and I can therefore not decisively argue that it can’t be successfully defended.

I will first describe what philosophical relativism is. I will then present two reasons why some people might find this theory plausible. Following this I will present my main criticism of philosophical relativism and then respond to three arguments a relativist might have against it. I will call these arguments “Bite the Bullet”, “Contextual relativism” and the “Preference argument”. Following this I will present another respons a relativist might have to counter the criticism. I will call this argument “cultural relativism”.

Philosophical relativism

Philosophical relativism is a theory that argues that every view and belief is equally legitimate - that they are on a par with each other and that you can’t prove that one belief is better or worse than the other. This theory is one of several ways you can go if you think that there are no objective moral truths, because you needn’t be a relativist to think that there are no moral objective truths. A philosophical relativist acknowledges, in an individual or a cultural ethical disagreement, that the other party’s view is on a par with his or hers and that they both have equally legitimate views. Even in a case of fundamental ethical disagreement the relativist would say that both views are equally legitimate.

Why some people find the theory plausible

First and foremost I would like to clarify that there is a difference between what is plausible and what is attractive. If someone finds a theory plausible they needn’t be attracted by it and vice versa. Furthermore, some people might find the theory plausible because they are attracted by it and vice versa.

Some people find the theory of philosophical relativism plausible because they have acknowledged that there don’t seem to be any moral objective truths.

Some people might also find the theory plausible because they have acknowledged that there don’t seem to be any decisive solutions to moral problems.

Criticism of philosophical relativism

One salient counterargument to the theory of philosophical relativism is that it can’t refute views that are morally wrong. If all views are to be seen as equally legitimate it would also allow for the justification of morally wrong actions. A philosophical relativist would have to say that the Nazi’s view on jews is equally legitimate as someone who holds a different view, since everyone is justified to hold their own views as long as it accords with their fundamental values. Below are three responses a relativist may have to counter the argument above:

Bite the bullet

Yes, the Nazi’s view on jews is a view that is as legitimate as any other to hold and all moralities are equal. There are no moral objective truths.

Contextual relativism

The meaning of legitimate, or of right and wrong, or of true or false is dependent on the context. The meaning of these terms are subject to change and the usage of them different from one culture to another, so I can therefore not argue that their view, back then, is morally right or wrong.

Preference argument

Even if someone is a relativist they can favor some views over others and have a strong inner feeling of what is right and wrong, only they are not entitled to interfere with other people’s views. They can’t prove that their view is somehow better than another view but they might still prefer one. So, they can say that they are strongly against how the Nazi’s viewed the jews but that they can’t prove that that view is better or worse than the Nazi’s.

The “Bite the bullet” argument above in defence of philosophical and semantic relativism seems to be a rather extreme argument that could allow for the justification of murder. The consequences of “Contextual relativism” is that the terms lose their meaning. If we can’t be certain that the term “true” means the same thing independent of context then we won’t be able to communicate with each other. The “Preference argument” does not really solve the problem with having to accept the justification of morally wrong actions because it still allows for them. Furthermore, it doesn’t allow for moral progress.

Cultural relativism

A philosophical relativist might also object that one culture should not interfere with another culture and that we should let each and every culture decide what is best for themselves. Why should we think that a culture who fosters a certain set of moral values, different from another culture’s values, are justified to judge or force their values upon the other? Maybe that other culture, by their measurements and standards of what is good and bad, are equally or more satisfied, or justified, to hold their beliefs than the first.

To this argument a person who rejects relativism may object that a person can be part of several different groups and cultures. These groups may also have totally different views and there may be difficulties in identifying which truths are relative to which groups. This means that the person who is part of several different groups may not know which norms or moral values they should adhere to.

I would all in all say that this counterargument to philosophical relativism is decisive but I can still imagine how a relativist might be able to respond to them (see above). We have also seen that the relativist's replies just lead to other problems.

***

3. Bentham's Arguments for Utilitarianism

Question

In Chapters 1 and 2 of his Principles of Morals and Legislation, Bentham offers three arguments for the claim that Utilitarianism is the correct moral theory. Describe each argument briefly, and then pick out the one that you believe is strongest. Explain that argument in the most persuasive way you can. Then assess the argument from a critical perspective. Raise at least one objection to it. Then consider how Bentham might reply. In the end, are you persuaded by that argument? Why or why not?

Introduction

In this essay I will first describe shortly what the utilitarian theory of morality consists of. Following this I will describe three arguments of Bentham’s that purport to show that Utilitarianism is the correct moral theory. These arguments will be called 1. “The semantic argument”, 2. “The argument from human nature” and 3. “The incoherence of rival views”. Then I will explain the third argument as persuasively as I can, since I find this argument to be the strongest of the three, and raise one objection to it. I will call this objection the “every moral theory” objection. Following this I will consider how Bentham may reply to my objection, and then explain why I find the “incoherence of rival views” argument persuasive.

Utilitarianism

Utilitarianism is a form of consequentialism which assesses actions as right or wrong based on their results. The utilitarian theory of morality says that, when we are faced with a choice of actions, we should choose the action that promotes the most net good (e.g happiness and pleasure), i.e: the action that maximizes utility when we have subtracted pain from pleasure. That action is said to be the only morally right action to take and all the others morally wrong.

The three arguments

Here are three of the arguments Bentham brings forth to support Utilitarianism.

1. The semantic argument

This argument is based around the notion of what the terms “right” and “wrong” mean. Bentham thought that the terms “right” and “wrong” couldn’t possibly have any other meaning than “productive of benefit” and “productive of harm”. Bentham doesn’t give us any argument in favor of this claim, rather he is asking us if we can argue for the meaning of these terms to be different, and if so, how? The argument also says that Utilitarianism is the only moral theory that uses these terms in that way.

2. The argument from human nature

A second argument Bentham brings forth appeals to the nature of humans. He states that the utilitarian theory of morality is the only theory that is in conformity with human nature. Bentham supports this claim by saying that we humans are all already applying the utilitarian principle in our daily lives (perhaps unconsciously and inconsistently); we tend to avoid pain and seek pleasure. He also points out that we humans naturally approve of actions that increase pleasure and that we disapprove of actions that increase pain.

3. The incoherence of rival views

The third argument says that all other ethical systems are tacitly relying on, or are covert applications of, utilitarianism or that they are just unsystematic collections of our likes and dislikes.

Of these three arguments I find the third argument to be the strongest and I will below explain it as persuasively as I can.

The incoherence of rival views

My view is this [1]: If morality is to have worthwhile meaning for us humans, then, it has to be connected to human well-being. Furthermore, I think morality needs to function as a tool for us humans to determine which actions to take and which actions to avoid; that is, have real world applications. Even though there might be other [2] moral theories, such as hedonism, based on this notion of morality, I would argue that these views are applications of Utilitarianism, and that Utilitarianism is the basis on which these other views rely on. I cannot prove my claim since I cannot take into account every possible moral theory, but Utilitarianism is in my opinion the most neutral and grounded moral theory based on this notion: take the action that promotes the most net good, nothing else.

If we look at hedonism or virtue ethics or deontology, and ask ourselves and their subscribers how they would argue in favor their moral theory, and how they would go about convincing others that their moral theory is the best, I think that their arguments would have to appeal to human well-being in some way or another. If they don’t, I don’t think that a lot of people would be interested in the theory. They have to say something along the lines of: “My moral theory is the best since it tells me to do this because this results in that…, and that thing is desirable, or good.... This I think, is the reason why Bentham makes his assertion that all other moral theories are just covert applications of Utilitarianism.

Below I will describe the “every moral theory” counter-argument.

The “every moral theory” counter-argument

A person who disagrees with the “incoherence with rival views” argument might say that the argument doesn't hold since we can never be certain that there are no other moral theories that don’t rely on utilitarian principles; and the “incoherence with rival views” argument says that all other theories are just applications of Utilitarianism.

How Bentham might reply

Bentham would perhaps say that yes, we can never be certain that we have exhausted every moral theory, but we can never be 100% certain of anything. So this counter-argument, even if it works as a counter-argument to refute the “incoherence of rival views” argument, doesn’t carry that much weight because it could be a counter-argument to every argument ever made. Rather, if we look at every moral theory that has been presented so far we can see that they all rely on utilitarian principles [3].

Am I persuaded by the argument in the end? Why, why not?

I am persuaded by the original “incoherence of rival views” argument even though I can see how people can argue both in favor of Bentham’s argument or in favor of the “every moral theory” counter-argument. The counter-argument speaks to me since I am a person who appreciates precision, but still, the “incoherence of rival views” argument is in my opinion the stronger of the two. This is because I have an innate feeling that we humans have to be able to apply our knowledge, our thoughts and our wisdom to the the world, and that morality has to be connected with human well-being if morality is to have any worthwhile meaning for us. I also think that we have to accept that we can’t (at least at the moment), be 100% certain of everything, because if we don’t, we won’t make any progress as individuals or as a society; we will constantly be stuck in theoretical outcomes and never be able to take action.

---

[1] In this essay I am solely focusing on moral theories that, in my eyes, promote the connection between morality and human well-being, in my eyes: hedonism, virtue ethics and deontology I am fully aware that one could reject my whole argument and my claims just by saying that there is no connection between them, but, as I said I am only focused on this specific area in this essay.

[2] Henceforth when I write “other moral theories” I am only focusing, as stated above, on moral theories, that in my eyes, promote the connection between morality and human well-being.

[3] Let’s take deontology and virtue ethics. Bentham may have replied that “to be a virtuous person is good, or desirable, only because it leads to either an inner feeling of well-being, or to your being viewed by others as a good person, or to you letting others have an opportunity to experience well-being. So, it’s not the virtues themselves that are good, but the good they result in.

Bentham would probably have argued in the same way when it comes to deontology. He may have replied “Well, the rules set for us to follow, will lead to us acting in the right way are relying on utilitarian principles”. I think he would have supported his claim by saying, as has been previously mentioned, that if you ask a deontologist why you should subscribe to deontology he would probably have to (perhaps unconsciously) appeal to human well-being.

***

4. Killing one to save five

Question

When is it morally okay to kill one person so as to prevent five people from being killed?

- How would an act consequentialist think about this question?

What would an act consequentialist say about the cases we considered in Lecture 13?

- Does the act consequentialist get these cases right?

- If not, what is the right general account of when it is morally okay to kill one person so as to save five?

Introduction

In this essay I will look at how an act consequentialist (henceforward called “AC”) would think about questions and thought experiments that involve killing one so as to save five other people from being killed. I will also argue that it may be possible to come up with a general principle for when it would be morally okay to kill one so as to save five so long as we don’t mix in our intuitions in the discussion.

When is it morally okay to kill one person to save five people from being killed?

Depending on which moral theory one is subscribed to and which moral intuitions one might have one could respond differently to this question. Some people say that you should never kill one to save five, while others say that you should always do it. Some people are not sure and for some it varies.

Below I will shortly describe act consequentialism and then see how an act consequentialist (“AC”) may reply to this question.

Act consequentialism

Act consequentialism is a form of consequentialism which says that the only morally right action to take is the one that produces the most the net-good (the total bad of everyone subtracted from the total good of everyone), when compared with the net-good of other actions the agent can take.

An “AC”’s response to the question “When is it morally okay to kill one to save five from being killed?”

Given the following presuppositions: Each life in this situation counts for an equal amount of, let’s call them utility-points. The situation takes place in a closed universe, that is, we neglect everything else that has not been explicitly said to be part of the situation, e.g, people that might judge your acting. Given these presuppositions an “AC” would always say that you should kill the one person to save the five from being killed, since it’s the action that results in the most net-good compared with the action of not doing anything and letting five people die.

Below I will explain 3 thought experiments which all are based on, and are modified versions of, the question given above. I will first describe each of them and then see how an “AC” may reply (still given the presuppositions above). These thought experiments combined with the solutions an “AC” gives to them aim to evoke doubts about act consequentialism.

Different cases to test act consequentialism

Blood

You are a doctor and you have an anesthetized patient on a table. Five of your other patients need one liter blood each to survive and the anesthetized patient could provide it, but he’d die in the process of donating the blood. Should you suck the blood out of the anesthetized patient (thereby killing him) and save the five others that would otherwise die?

Trolley Problems

Footbridge

A trolley is heading toward five persons on a track and the driver has lost control over it. You and a large fellow are standing on a footbridge that goes over the track. By knocking the large fellow off the bridge and causing him to fall onto the track and die when the trolley hits him, you could save the five persons that would otherwise have died by the trolley (the large fellow slows it down). Should you knock him over?

Switch

A trolley is heading toward five people and the driver has lost control. You are now instead standing next to a switch and you can pull the switch causing the trolley to change tracks. On the other track there’s another person. Should you pull the switch and save five persons that would otherwise have been killed but kill the one on the other track or don’t and let the five persons die?

Does the act consequentialist get the cases right?

Given the same presuppositions as above an “AC” would always kill one so as to save five others from being killed. I take it your immediate reaction to this is that the “AC” acts wrongly by killing the large fellow, but you may feel that it seems okay (or at least not wrong) to pull the switch. Personally I’m not entirely sure, but I still feel as though the act consequentialist’s way of acting is a bit radical.

Is it possible to come up with a general principle for when it’s okay to kill one to save five? A general principle that sorts the cases above in the “right” way; not pushing the large fellow off the bridge (Footbridge), but still maybe allowing for one to pull the switch (Switch)?

General moral principle for when it’s ok to kill one so as to save five?

My view is that we can come up with a general principle for the cases above. Maybe that principle encompasses all of our intuitions and gut feelings, maybe not, and my point is that it doesn’t necessarily have to do that. Why should our intuitions set the standard for plausibility or for morally justifiable actions?

We humans have evolved to act in certain ways in certain situations. Why rule out a general principle such as “take the action that promotes the most net-good; in the cases above, kill one to save five.” based on our intuitions? Our intuitions are not only unreliable but they are also inconsistent. Our intuitions of what’s morally right or wrong might change over time and acting on them doesn’t always lead to the “best” outcome.

The reason as to why we humans won’t push the large fellow off the bridge is due to evolution. When we are standing on the footbridge we are unconsciously thinking about how our acting will be viewed upon by others, even if we can’t see anyone there. Maybe there’s someone there, only you can’t see him. If, on the off-chance there is someone there judging your acting, you would be worse off pushing the large fellow than not pushing him since the watcher may then view you as untrustworthy. In the past you’d not want to risk getting alienated from your group.

To say that it is impossible to come up with a general principle for the cases above, or to rule out theories or principles, based on our intuitions, I find short-sighted.

Liberal Democracy and Epistemic Neutrality

Utdrag ur:

Klemens Kappel, Liberal Democracy and Epistemic Neutrality (first draft)


”...liberal democracy does not require neutrality with respect to conceptions of justice. In just the same way, one might suggest, does liberal democracy not require epistemic neutrality. To expand Kymlicka’s terminology, neutrality is only required between conceptions of the good which are both justice-respecting and fact-respecting. The general idea would be that the state (and societal culture, to some extent) is obliged to remain neutral between different conceptions of the good, or sets of individual values, but not between different different conceptions of what the facts are. [...]

Suppose that the state adopts certain policies based on controversial views of the facts. For example, suppose the state refuses to fund alternative medicine in public health care because of the lack of evidence for efficacy, or refuses to include intelligent design in school curricula on the ground that intelligent design lack[s] proper scientific basis, or launches an educational initative to convice the population that political action is needed in response to climate change, [...] thereby implying that climate deniers lack proper scientific grounding. Individuals who disagree with these decisions, but are nonetheless forced to comply with them in the relevant sense, have no ground for complaint, then, because the requirement of value neutrality does not extend that far. Those who firmly believe in the efficacy of alternative medicine, for example, do not have a just complaint that it is not available in a publicly funded health care system. Or put it differently: people have a right to hold and act upon the belief that alternative medicine is highly effective, of course, but they must themselves bear the cost of these beliefs, and they have no ground for complaint if public policy is based on incompatible factual beliefs.

Now, I have merely indicated how one might in principle respond to the fact that certain factual beliefs may play a role for individual autonomy similar to that of individual values. I haven’t argued that liberal democracy should adopt epistemically non-neutral attitudes (though I believe that it should). I have merely indicated how epistemic non-neutrality is a possibility within the general framework. Moreover, I haven’t argued that the liberal state should side with best science, when [it] adopts policies in areas of factual controversies, and commits itself to non-neutral stances on these controversies (though this is also what I think the state should). In fact, it is useful to have a general term for this particular very familiar non-neutral attitude, so let me simply refer to it as scientific non-neutrality.”

s. 23-24


”...we need not view liberal democracy as wedded to any general principles of epistemic neutrality, beyond what is required by our basic cognitive freedoms. This means that contrary to what is asserted by many commentators, there is [no] inherent conflict between democracy and the inherent non-neutrality of science. Hence, Seife and others are wrong to think that science and democracy is necessarily at odds. Liberal democracy can adopt the inherent non-neutrality of science. This is not yet to hold that liberal democracy ought to embrace scientific non-neutrality. I haven’t argued that any particular epistemically non-neutral stance adopted by the state or by societal culture is justified, or how one should state a satisfactory defense of a non-neutral policy, though I do think we should endorse what I called scientific non-neutrality. I have indicated that even if the best arguments for scientific non-neutrality are epistemically circular, [that] does not prevent us from rationally endorsing scientific non-neutrality.”

s. 26


Ur:

Klemens Kappel, Liberal Democracy and Epistemic Neutrality (first draft)

https://www.academia.edu/1755455/Liberal_Democracy_and_Epistemic_Neutrality

Metzinger's dangerous idea

Metzinger verkar vara orolig för vad som skulle hända om folk i allmänhet insåg att de inte har fri vilja. Han kan nog ha rätt i att illusionen är tätt sammankopplad med utvecklingen av samarbete och moraliska intuitioner; och också i att självbild och kulturell kontext påverkar varandra; men jag tror och hoppas likväl att vi nu är mogna att lämna retributivism och andra stenåldersidéer bakom oss, och att detta kan göras utan att samhället regredierar -- snarare tvärtom!

Imagine that we have created a society of robots. They would lack freedom of the will in the traditional sense, because they are causally determined automata. But they would have conscious models of themselves and of other automata in their environment, and these models would let them interact with others and control their own behavior. Imagine that we now add two features to their internal self- and other-person models: first, the erroneous belief that they (and everybody else) are responsible for their own actions; second, an "ideal observer" representing group interests, such as rules of fairness for reciprocal, altruistic interactions. What would this change? Would our robots develop new causal properties just by falsely believing in their own freedom of the will? The answer is yes: moral aggression would become possible, because an entirely new level of competition would emerge -- competition about who fulfills the interests of the group best, who gains moral merit, and so on. You could now raise your own social status by accusing others of being immoral or by being an effective hypocrite. A whole new level of optimizing behavior would emerge. Given the right boundary conditions, the complexity of our experimental robot society would suddenly explode, though its internal coherence would remain. It could now begin to evolve on a new level. The practice of ascribing moral responsibility -- even if based on delusional PSMs [Phenomenal Self Models] -- would create a decisive, and very real, functional property: Group interests would become more effective in each robot's behavior. The price for egotism would rise. What would happen to our experimental robot society if we then downgraded its members' self-models to the previous version -- perhaps by bestowing insight?

[...]

 Neuroscientists like to speak of "action goals", processes of "motor selection", and the the "specification of movements" in the brain. As a philosopher (and with all due respect), I must say that this, too, is conceptual nonsense. If one takes the scientific worldview seriously, no such things as goals exist, and there is nobody who selects or specifies an action. There is no process of "selection" at all; all we really have is dynamic self-organization. Moreover, the information-processing taking place in the human brain is not even a rule-based kind of processing. Ultimately, it follows the laws of physics. The brain is best described as a complex system continuously trying to settle into a stable state, generating order out of chaos.

 According to the purely physical background assumptions of science, nothing in the universe possesses an inherent value or is a goal in itself; physical objects and processes are all there is. That seems to be the point of the rigorous reductionist approach -- and exactly what beings with self-models like ours cannot bring themselves to believe. Of course, there can be goal representations in the brains of biological organisms, but ultimately -- if neuroscience is to take its own background assumptions seriously -- they refer to nothing. Survival, fitness, well-being, and security as such are not values or goals in the true sense of either word; obviously only those organisms that internally represented them as goals survived. But the tendency to speak about the "goals" of an organism or a brain makes neuroscientists overlook how strong their very own background assumptions are. We can now begin to see that even hardheaded scientists sometimes underestimate how radical a naturalistic combination of neuroscience and evolutionary theory could be: It could turn us into beings that maximized their overall fitness by beginning to hallucinate goals.

I am not claiming that this is the true story, the whole story, or the final story. I am only pointing out what seems to follow from the discovery of neuroscience, and how these discoveries conflict with our conscious self-model. Sub personal self-organization in the brain simply has nothing to do with what we mean by "selection". Of course, complex and flexible behaviors caused by inner images of "goals" still exist, and we may also continue to call these behaviors "actions". But even if actions, in this sense, continue to be a part of the picture, we may learn that agents do not -- that is, there is no entity doing the acting. 

Thomas Metzinger, The Ego Tunnel, 2009, p. 129-131

Intentional agents as leaky abstractions

I have spent the day reading Kaj Sotala's sequence "Multiagent Models of Mind" on LessWrong. Six posts are published so far. At least three more are planned; they look really interesting, and I hope that they will answer my main question so far:

What does this actually mean, and what is the motivation for saying it?

Agent-ness being a leaky abstraction is not exactly a novel concept for Less Wrong; it has been touched upon several times, such as in Scott Alexander’s Blue-Minimizing Robot Sequence. At the same time, I do not think that it has been quite fully internalized yet, and that many foundational posts on LW go wrong due to being premised on the assumption of humans being agents. In fact, I would go as far as to claim that this is the biggest flaw of the original Sequences: they were attempting to explain many failures of rationality as being due to cognitive biases, when in retrospect it looks like understanding cognitive biases doesn’t actually make you substantially more effective. But if you are implicitly modeling humans as goal-directed agents, then cognitive biases is the most natural place for irrationality to emerge from, so it makes sense to focus the most on there.

This was what piqued my interest in reading, and what kept me going for five hours straight (!) But I didn't find what I was looking for. I did get a lot of other useful information, though: All in all, this is a wonderful, brilliant, deep and highly thought-provoking text. A lot of work and thought has gone into it. Kaj is surely one of the brightest minds around.

Now, I have no problem whatsoever with the first part of the passage quoted above. Of course intentional agents are an abstraction, and as such of course it is leaky. My concern lies with the second part: It seems to suggest that viewing people as intentional agents is mistaken; to coarse a model; misleading. Which seems to lead to the conclusion that cognitive biases are the wrong way to characterize human thinking and behavior. Which suggests that they are not even real...

I may be overly trigger-happy here. I am not out to criticize Sotala himself, nor - as it turns out - any major part of what he actually has written in this sequence so far. It is just that I am currently (yet again) in a process of investigation and possible re-orientation of what I believe is best characterized as the latest round of the "rationality wars". I am currently reading Gerd Gigerenzer's book "Risk Savvy", in a long stretch of similar stuff (e.g. Mercier & Sperber), with the aim of trying to reconcile the seemingly opposing sides in an ever ongoing battle for the right to define "rationality".

I am a long-time fan of Kahneman (and Dennett). It may be self-delusion on my part, but much of the criticism leveled at him and others seem to me either plain wrong, ideologically motivated or mistaken. The more I read, the more I get the feeling that my intuitive interpretation of Kahneman and others does not need updating; rather, it is his critics who either straw-man him or just do not have the whole picture.

Sotala promises to tell me why the biases and fallacies school of thought is lacking. But I just don't see it.

I would go as far as to claim that this is the biggest flaw of the original Sequences: they were attempting to explain many failures of rationality as being due to cognitive biases, when in retrospect it looks like understanding cognitive biases doesn’t actually make you substantially more effective. But if you are implicitly modeling humans as goal-directed agents, then cognitive biases is the most natural place for irrationality to emerge from, so it makes sense to focus the most on there. 

Just knowing that an abstraction leaks isn’t enough to improve your thinking, however. To do better, you need to know about the actual underlying details to get a better model. In this sequence, I will aim to elaborate on various tools for thinking about minds which look at humans in more granular detail than the classical agent model does. Hopefully, this will help us better get past the old paradigm.

There is a sense in which I get this: Higher-level abstractions trade accuracy for expediency, yes. And sometimes you need to go down an explanatory level or two, depending on your goals. But when it comes to explaining human decision making, or how humans view themselves and others and the society that emerges from their interaction, or why this is the case, or how and when problems and contradictions occur, or what to do about it... Well, I just don't see the need to shed the intentional stance or to deconstruct it. (Apart from convincing people that they are usually MoreWrong than they think.)

My main question when reading the above was - and still is: Are we talking descriptively, prescriptively, or normatively?

Mercier & Sperber, for instance, accuse Kahneman of assuming a logical, but flawed, human psyche. Rationality to them seems to mean a description of what people are, what they have evolved into - even if the process is incomplete. They then go on to sarcastically point out that there is no evolutionary reason to expect people to be logical inference machines. At the same time they redefine rationality to mean "socially flexible and pragmatic" rather than logical, and to contend that that is exactly what people are - so stop shaming them for not being able to solve logical puzzles. Also, they and Gigerenzer and others go on to say: "Oh, and by the way, people are quite adept at logic, statistics and probabilities - if you just stop tricking them!"

To me, this is highly confused. Man is not the measure of everything. Rationality, meaning logic, statistical thinking, utility maximizing etc, is a cultural invention, a norm, a standard to which we aspire - and should aspire. The fact that we have a hard time living up to those ideals is an observation of facts, and there are plenty of good reasons why this is the case. But it is equally obvious that we should do whatever we can to get beter at it. We can't (yet) re-engineer ourselves, so we need to work on education, societal structures, political systems etc.

Gigerenzer thinks Sunstein is an autocrat who doesn't trust people to know their own good. I think that Sunstein is way to libertarian.


---

One line of evidence for this are subliminal priming experiments, not to be confused with the controversial “social priming” effects in social psychology; unlike those effects, these kinds of priming experiments are well-defined and have been replicated many times.

Is there a difference? Sotala never uses the term, but I constantly think of associative networks (and perceptrons). Priming is potential for spreading activation. Priming is priming, however mixed results and exaggerated results from sloppy social-psychology experiments.



---

First, in order for the robot to take physical actions, the intent to do so has to be in its consciousness for a long enough time for the action to be taken. If there are any subagents that wish to prevent this from happening, they must muster enough votes to bring into consciousness some other mental object replacing that intention before it’s been around for enough time-steps to be executed by the motor system. (This is analogous to the concept of the final veto in humans, where consciousness is the last place to block pre-consciously initiated actions before they are taken.)

Oh, oh, oh! Veto without a libertarian prime mover. Yes! This resolves the tension I experienced when reading Patrik Lindenfors' speculations on free will in his new book "The Cultural Animal". Libet-experiments should measure several different signals simultaneously.


---

Second, the different subagents do not see each other directly: they only see the consequences of each other’s actions, as that’s what’s reflected in the contents of the workspace. In particular, the self-narrative agent has no access to information about which subagents were responsible for generating which physical action. It only sees the intentions which preceded the various actions, and the actions themselves. Thus it might easily end up constructing a narrative which creates the internal appearance of a single agent, even though the system is actually composed of multiple subagents.

Oh! Self-serving bias, confabulation, FAE, myside bias... But what is the difference in practice? It is still a case of self-deception.


---

Third, even if the subagents can’t directly see each other, they might still end up forming alliances. For example, if the robot is standing near the stove, a curiosity-driven subagent might propose poking at the stove (“I want to see if this causes us to burn ourselves again!”), while the default planning system might propose cooking dinner, since that’s what it predicts will please the human owner. Now, a manager trying to prevent a fear model agent from being activated, will eventually learn that if it votes for the default planning system’s intentions to cook dinner (which it saw earlier), then the curiosity-driven agent is less likely to get its intentions into consciousness. Thus, no poking at the stove, and the manager’s and the default planning system’s goals end up aligned. 

Fourth, this design can make it really difficult for the robot to even become aware of the existence of some managers. A manager may learn to support any other mental processes which block the robot from taking specific actions. It does it by voting in favor of mental objects which orient behavior towards anything else. This might manifest as something subtle, such as a mysterious lack of interest towards something that sounds like a good idea in principle, or just repeatedly forgetting to do something, as the robot always seems to get distracted by something else. The self-narrative agent, not having any idea of what’s going on, might just explain this as “Robby the Robot is forgetful sometimes” in its internal narrative.

Ah! Dunning-Kruger, ignorance, witness psychology, unwarranted self-assurance... But what is the difference from intentional agents and the bias and fallacies perspective?



---

Fifth, the default planning subagent here is doing something like rational planning, but given its weak voting power, it’s likely to be overruled if other subagents disagree with it (unless some subagents also agree with it). If some actions seem worth doing, but there are managers which are blocking it and the default planning subagent doesn’t have an explicit representation of them, this can manifest as all kinds of procrastinating behaviors and numerous failed attempts for the default planning system to “try to get itself to do something”, using various strategies. But as long as the managers keep blocking those actions, the system is likely to remain stuc

Aha! Akrasia, ”irrationality” in the sense of not living up to the homo economics template etc...


---

Sixth, the purpose of both managers and firefighters is to keep the robot out of a situation that has been previously designated as dangerous. Managers do this by trying to pre-emptively block actions that would cause the fear model agent to activate; firefighters do this by trying to take actions which shut down the fear model agent after it has activated. But the fear model agent activating is not actually the same thing as being in a dangerous situation. Thus, both managers and firefighters may fall victim to Goodhart’s law, doing things which block the fear model while being irrelevant for escaping catastrophic situations.”

But this is missing an evolutionary perspective (which Sotala brings up much later) outside of the individual agent. Systems that are reasonably well adjusted beget offspring with pre-installed settings that also work reasonably well (as long as the environment doesn't change too much).

Goodhart! Yes! Isn't that the perfect summation of every bias in the book!?


It's a (too) tall order to get people to change their evolved picture of themselves and others, from intentional agents to more or less coordinated subsystems.

Normatively, also we want to act, judge and be judged as intentional agents.



---

Exiles are said to be parts of the mind which hold the memory of past traumatic events, which the person did not have the resources to handle. They are parts of the psyche which have been split off from the rest and are frozen in time of the traumatic event. When something causes them to surface, they tend to flood the mind with pain. For example, someone may have an exile associated with times when they were romantically rejected in the past. 

IFS further claims that you can treat these parts as something like independent subpersonalities. You can communicate with them, consider their worries, and gradually persuade managers and firefighters to give you access to the exiles that have been kept away from consciousness. When you do this, you can show them that you are no longer in the situation which was catastrophic before, and now have the resources to handle it if something similar was to happen again. This heals the exile, and also lets the managers and firefighters assume better, healthier roles.

Very Freudian! Both in a good sense and in a bad one. (And I suspect that the IFS crowd really longs for a true Self - which is exactly what there isn't!)


---

In my earlier post, I remarked that you could view language as a way of joining two people’s brains together. A subagent in your brain outputs something that appears in your consciousness, you communicate it to a friend, it appears in their consciousness, subagents in your friend’s brain manipulate the information somehow, and then they send it back to your consciousness. 

If you are telling your friend about your trauma, you are in a sense joining your workspaces together, and letting some subagents in your workspace, communicate with the “sympathetic listener” subagents in your friend’s workspace. So why not let a “sympathetic listener” subagent in your workspace, hook up directly with the traumatized subagents that are also in your own workspace?

Yeah... This is what Mercier & Sperber get right - social cognition. But a bit too idealized on communication with others. There is a lot of "pollution" in those exchanges... Even internal monologues are polluted by irrelevant concerns and noise.


---

Instead of remaining blended, you then use various unblending / cognitive defusion techniques that highlight the way by which these thoughts and emotions are coming from a specific part of your mind. You could think of this as wrapping extra content around the thoughts and emotions, and then seeing them through the wrapper (which is obviously not-you), rather than experiencing the thoughts and emotions directly (which you might experience as your own). 

...when I became aware of how much time I spent on useless rumination while on walks, I got frustrated. And this seems to have contributed to making me ruminate less: as the system’s actions and their overall effect were metacognitively represented and made available for the system’s decision-making, this had the effect of the system adjusting its behavior to tune down activity that was deemed useless.

Creativity? Heureka moments? Openness to new impressions? (This is discussed later.)

---

Similarly, several of the experiments which get people to exhibit incoherent behavior rely on showing different groups of people different formulations of the same question, and then indicating that different framings of the same question get different answers from people. It doesn’t work quite as well if you show the different formulations to the same people, because then many of them will realize that differing answers would be inconsistent.

This is the point of contention in the rationality wars!


---

The original question which motivated this section was: why are we sometimes incapable of adopting a new habit or abandoning an old one, despite knowing that to be a good idea? And the answer is: because we don’t know that such a change would be a good idea. Rather, some subsystems think that it would be a good idea, but other subsystems remain unconvinced. Thus the system’s overall judgment is that the old behavior should be maintained.

Yees! But normatively, we can know that something is better, while emotionally we do not experience it that way. This is what a bias is!



---

Nevertheless, a fundamental problem remains: at any point in time, which mode should be allowed to control which component of a task? Daw et al. have used a computational approach to address this problem. Their analysis was based on the recognition that goal-directed responding is flexible but slow and carries comparatively high computational costs as opposed to the fast but inflexible habitual mode. They proposed a model in which the relative uncertainty of predictions made by each control system is tracked. In any situation, the control system with the most accurate predictions comes to direct behavioural output. 

Note those last sentences: besides the subsystems making their own predictions, there might also be a meta-learning system keeping track of which other subsystems tend to make the most accurate predictions in each situation, giving extra weight to the bids of the subsystem which has tended to perform the best in that situation. We’ll come back to that in future posts.

Automatic vs controlled processes (system 1 and 2). Again, a tall order to transition from the former to the latter. Energy conservation. But also, built-in inertia to avoid paralysis (see Minsky quote):

”Human self-control is no simple skill, but an ever-growing world of expertise that reaches into everything we do. Why is it that, in the end, so few of our self-incentive tricks work well? Because, as we have seen, directness is too dangerous. If self-control were easy to obtain, we'd end up accomplishing nothing at all.”

---

When there is significant uncertainty, the brain seems to fall back to those responses which have worked the best in the past - which seems like a reasonable approach, given that intelligence involves hitting tiny targets in a huge search space, so most novel responses are likely to be wrong.

Also over evolutionary time, over generations. Bias as hard-coded patterns which have previously comprised the best compromise.

---

...positive or negative moods tend to be related to whether things are going better or worse than expected, and suggest that mood is a computational representation of momentum, acting as a sort of global update to our reward expectations.

Yeeesss!!!


---

So to repeat the summary that I had in the beginning: we are capable of changing our behaviors on occasions when the mind-system as a whole puts sufficiently high probability on the new behavior being better, when the new behavior is not being blocked by a particular highly weighted subagent (such as an IFS protector whose bids get a lot of weight) that puts high probability on it being bad, and when we have enough slack in our lives for any new behaviors to be evaluated in the first place. Akrasia is subagent disagreement about what to do.

This is perfectly in line with the bias perspective.


---

Likewise, the subagent frame seems most useful when a person’s goals interact in such a way that applying the intentional stance - thinking in terms of the beliefs and goals of the individual subagents - is useful for modeling the overall interactions of the subagents.

Confusing. Wasn't the whole point to question the intentional agent, the system as a whole, the unmoved mover, the green man at the center of it all?



---

More generally, subagents may be incentivized to resist belief updating for at least three different reasons (this list is not intended to be exhaustive): 

1 The subagent is trying to pursue or maintain a goal, and predicts that revising some particular belief would make the person less motivated to pursue or maintain the goal. 

2 The subagent is trying to safeguard the person’s social standing, and predicts that not understanding or integrating something will be safer, give the person an advantage in negotiation, or be otherwise socially beneficial. For instance, different subagents holding conflicting beliefs allows a person to verbally believe in one thing while still not acting accordingly - even actively changing their verbal model so as to avoid falsifying the invisible dragon in the garage. 

3 Evaluating a belief would require activating a memory of a traumatic event that the belief is related to, and the subagent is trying to keep that memory suppressed as part of an exile-protector dynamic.

Reminds me of Omohundro's thesis on goal-preservation. (And Olle's problematization of the same...)


---

Suppose that a disease, or a monster, or a war, or something, is killing people. And suppose you only have enough resources to implement one of the following two options:

1. Save 400 lives, with certainty.
2. Save 500 lives, with 90% probability; save no lives, 10% probability.

Most people choose option 1. [...] If you present the options this way:

1. 100 people die, with certainty.
2. 90% chance no one dies; 10% chance 500 people die. 

Then a majority choose option 2. Even though it's the same gamble. You see, just as a certainty of saving 400 lives seems to feel so much more comfortable than an unsure gain, so too, a certain loss feels worse than an uncertain one. 

In my previous post, I presented a model where subagents which are most strongly activated by the situation are the ones that get access to the motor system. If you are hungry and have a meal in front of you, the possibility of eating is the most salient and valuable feature of the situation. As a result, subagents which want you to eat get the most decision-making power. On the other hand, if this is a restaurant in Jurassic Park and a velociraptor suddenly charges through the window, then the dangerous aspects of the situation become most salient. That lets the subagents which want you to flee to get the most decision-making power. 

Eliezer’s explanation of the saving lives dilemma is that in the first framing, the certainty of saving 400 lives is salient, whereas in the second explanation the certainty of losing 100 lives is salient. We can interpret this in similar terms as the “eat or run” dilemma: the action which gets chosen, depends on which features are the most salient and how those features activate different subagents (or how those features highlight different priorities, if we are not using the subagent frame). 

Suppose that you are someone who was tempted to choose option 1 when you were presented with the first framing, and option 2 when you were presented with the second framing. It is now pointed out to you that these are actually exactly equivalent. You realize that it would be inconsistent to prefer one option over the other just depending on the framing. Furthermore, and maybe even more crucially, realizing this makes both the “certainty of saving 400 lives” and “certainty of losing 100 lives” features become equally salient. That puts the relevant subagents (priorities) on more equal terms, as they are both activated to the same extent. 

What happens next depends on what the relative strengths of those subagents (priorities) are otherwise, and whether you happen to know about expected value. Maybe you consider the situation and one of the two subagents (priorities) happens to be stronger, so you decide to consistently save 400 or consistently lose 100 lives in both situations. Alternatively, the conflicting priorities may be resolved by introducing the rule that “when detecting this kind of a dilemma, convert both options into an expected value of lives saved, and pick the option with the higher value”. 

By converting the options to an expected value, one can get a basis by which two otherwise equal options can be evaluated and chosen between. Another way of looking at it is that this is bringing in a third kind of consideration/subagent (knowledge of the decision-theoretically optimal decision) in order to resolve the tie.

1. 400 survivors is not interpreted as 100 deaths, but rather as "at most 100 deaths".

2. This is Gigerenxer's schtick: "We don't really have any biases. It's just a question of presenting or rephrasing situations so that it becomes obvious how to deal with them." But it is precisely the fact that this is not done which comprises the bias! (That, and the fact that we don't even experience any need to rephrase the situation.)

3. What is the rationale for preferring expected utility over, say, a sure positive? How does one resolve that conflict, before and after the choice? To oneself? To others?


---

The structure of the “parking ticket” and “cheque” scenarios are equivalent, in that both cases you can take an action to be $90 better off after 30 days. If you notice this, then it may be possible for you to re-interpret the action of paying off the parking ticket as something that gains you money, maybe by something like literally looking at it and imagining it as a cheque that you can cash in, until cashing it in starts feeling actively pleasant.

No. In one case, you lose something, or end up owing something that you may not even have. You wouldn't survive if you had to give away your food. In the other case, you go from surviving as usual to receiving a windfall, an extra bonus. This is exactly the kind of ill-conceived homo economicus rationality that even the economists have abandoned.


---

Reading through my notes, I am starting to wonder if what you're really saying is this: "There is no man in the middle, no unmoved mover, no central control to which we can ascribe beliefs and desires, or hold accountable; who is the author of our destiny, the locus of our (free) will."

And of course I agree.

Maybe your point is that viewing ourselves and others as intentional agents create or reinforce these misconceptions; that we need to understand that we *don't* actually have good reasons for thinking, feeling and doing what we do. That to humble ourselves, we need to understand that the self, the agent, is a figment of our imagination, an illusion to explain our subconscious elephant to our translucent rider...

And I agree.

But still: The best way to summarize the totality is the intentional agent. Maybe this is the reason why I am confused: I have always, ever since childhood, been perfectly onboard with a super-cynical view of people as biological contraptions, recently endowed with (an experience of) (self-)awareness; trying to make sense of the (apparent) voices inside our heads.

The intentional agent is a big improvement over many previous centuries of an over-inflated sense of importance. It is a description of how we have evolved to navigate in the world and coordinate with other moving objects. It is an "as if"-model. Nothing more. This is blatantly obvious to me.

The biases-and-fallacies paradigm serves as an educational device in the service of convincing people who think that we know what we (and others around us) are doing, that we don't. Or at least, that our guesses are just that: shortcuts that try to minimize catastrophic failures in a maximum number of (familiar) situations.