Journal of Childhood & Developmental Disorders

Introduction

One of the major problems with current concepts used in psychiatry is the basic assumptions on which much psychiatric research rests. In order to scientifically evaluate the proposition that there is a natural category of dysfunction/disorder called, for example, ‘Attention Deficit Hyperactivity Disorder’ (ADHD), we must start with the null hypothesis. That is we must assume, until proven otherwise, that there is no characteristic measurable feature corresponding with what we are defining as ADHD and set out to demonstrate this null hypothesis cannot be true [1]. This is a foundational assumption behind the development of knowledge through the scientific method. It requires the investigator state their hypothesis in negative terms, forming a null hypothesis, which predicts that there will be no difference between the test groups. In this example, the null hypothesis assumes that there is no characteristic identifiable, measurable difference between those labelled ADHD and a comparable population (e.g. in terms of age, gender, learning ability etc.). Applying scientific principles through this basic methodology is the basis for the construction of Evidence Based Medicine (EBM). Until we have demonstrated that this null hypothesis can’t be true, then research that simply assumes that ADHD (for example) is a scientifically validated construct with explanatory powers relies on unsupported assumptions and so belongs to an unscientific paradigm [2]. Unfortunately most mainstream publications in psychiatry, including those about autism, fall into this category and builds knowledge on foundations that have more to do with wishful thinking and marketing than science.

The corruption of these principles of EBM has been widely debated. For example, pharmaceutical companies have been able, through a variety of strategies, to bias the evidence base towards the products they make [3,4]. EBM has fallen victim to broader political forces, leading to collusion between profit focussed organisations and professional guilds such as the American Psychiatric Association [5]. Thus profit making and guild interests often triumph over science.

In addition, whilst use of the positivist, hypothesis testing, measurement focussed pursuit of objective, value free knowledge about the world ‘out there’ (beyond our imaginations) works well for systems and phenomena governed by ‘laws of nature’, it is not always the most appropriate method for a full understanding of subjective, meaning generating conscious life. Kidney’s don’t have dreams, intentions, or agonise over the meaning of their existence, and so we can use positivist, empirical methods to study them. A medicine of the mind is therefore different to medicine of the Kidneys [6].

I believe the task of critically examining current assumptions for their scientific legitimacy as well as broader engagement with cross disciplinary perspectives is an urgent one for psychiatrists and psychologists. We have little evidence that outcomes after treatment in standard mental health services are improving and much to suggest that in countries with the most developed mental health services outcomes may well be deteriorating [7- 10]. The disconnect between academic and clinical endeavours in psychiatry is now so acute that some leading figures have suggested that without reform, academic psychiatry will become redundant in a few decades [11]. This article is thus written in that spirit and focuses on the necessity to critically examine an aspect of practice that has become widespread and often accepted without question – the use of autism ‘diagnostic’ tools. To do this, I critically appraise a two day training course in the use of Autism Diagnostic Observation Schedule (ADOS) that I attended.

Is autism a scientific construct?

Autism and Autism Spectrum Disorders (ASD), in mainstream clinical and academic endeavours are good examples of how marketing and wishful thinking have meant that a variety of unscientific assumptions have become accepted as facts of nature whereas, based on current available evidence, they can only qualify as facts of culture [12].

Reflecting on Practice

When I was training as a child psychiatrist in the early to mid 1990’s I came across two children diagnosed with autism in the whole of my 4 years of training placements. Both had marked functional impairments and had to attend specialist schools. I now encounter children and youth diagnosed with an ASD on a daily basis in clinical practice. The first epidemiological study on autism was published in 1966 and arrived at a prevalence figure for autism of 4.5 per 10,000 [13]. This figure has now expanded into a condition said to potentially affect 160 per 10,000 (1.6%) of the population [1] an over 3500% increase in prevalence in just 4 decades.

On what Basis did such an Extraordinary Expansion Happen?

The science

As I have discussed elsewhere the radical change in thinking about autism and what it is or isn’t came about through ideological changes not scientific discoveries. Genetic and brain imaging studies have failed to identify any specific replicable differences, abnormalities, or biomarkers [14,15]. In Timimi and McCabe we provide an overview of the evidence (or rather lack of it) supporting the idea that what we are calling ASD cannot, based on available scientific evidence, be called a diagnosis as there is no characteristic, identifiable, and measurable difference between those labelled with ASD and a comparable population. ASD is therefore a descriptive not explanatory classification. Below is a brief summary of our review. Belatedly, some, including prominent ASD researchers, are reaching the same conclusion.

The problem of heterogeneity: One of the immediate problems that autism researchers have is reaching a definition of ‘caseness’ that makes possible research that is likely to identify uniquely ‘autistic’ pathology. The more ‘fuzzy’ the boundaries the less likely you will capture a specific population that makes specific discoveries more likely. As noted there has been a roughly 3500% increase in the assumed prevalence of ASD, thus making the potential study population more and more heterogeneous. This heterogeneity in what is now considered as ‘autistic’ means that not only do what are considered ‘core’ features (such as difficulties in social communication) have a large cross over with people who would not consider themselves as having a ‘disorder’, but autism as a mental state and elements of autism symptoms as traits, are commonly found in a variety of other mental health categories from ADHD to conduct disorders and from attachment disorders to depression and anxiety.

Lack of molecular genetic findings: The assumed heritability of autism, as with many other psychiatric conditions, has been based on twin and family studies that are unable to disentangle environmental from genetic contributions and can therefore be discounted as providing reliable evidence of heritability [16,17]. Given the advancement of whole genome decoding technology, a vast amount of molecular genetic research has been conducted that can shed light on actual genetic contributions. Accumulation of studies means we have molecular genetic data on thousands of individuals ‘diagnosed’ with ASD. So far nothing with any level of meaningful significance has been found. The continuing failure to identify genetic specifics seems to have resulted in the majority of the human chromosome being identified as potentially harbouring autism genes, with prominent reviews typically concluding: “With the advent of next generation sequencing techniques, the number of genes found that are associated with ASD is increasing to over 800 genes; consequently, it is becoming even more challenging to find unified explanations and functional associations between the genes involved” [18].

Instead of facing up to the possibility that genes are not revealing themselves because they are not there, we have instead moved into an era where multiple research teams come together to create banks of ‘big data’ in the hope that this can reveal tiny associations. The most likely reason for the failure to find anything of any likely pathological significance is that there is nothing there. The null hypothesis thus stands – there are no specific genetic abnormalities or patterns associated with ASD.

Lack of neuroimaging findings: Autism brain research has had the same problem of replication found in the molecular genetic studies. Theories come and go and what, how, or where the apparent neurodevelopmental abnormality is, remains a mystery. Recently some researchers are giving up on the notion they will find anything specific connected to our current concept of ASD. Thus a 2016 paper entitled ‘ASD validity’, which includes the renowned autism researcher, professor Gillberg, amongst its authors, concludes, “The findings reviewed indicate that the ASD diagnosis lacks biological and construct validity” [19] and they recommend disbanding ASD diagnoses as a basis for research. Sadly, they go on to suggest a broader neurodevelopmental construct instead, without a hint of concern that the reasons for the failure of autism research’s capacity to reveal anything will likely continue with their even more heterogeneous construct. The null hypothesis thus stands – there are no specific brain abnormalities or patterns associated with ASD.

Autism Diagnostic Observation Schedule (ADOS)

The marketization of ASD has resulted in the growth of so called ‘tests’ for ASD. These tests are either observational or questionnaire based. They do not provide any physical data on the internal functioning of the body, and so cannot be viewed as having the same status as medical tests that measure physical features and that act as an aid to diagnosis. The ADOS is one of the most widely used of these ASD ‘tests’. In 2017 I attended, along with 13 other participants, two day training in the ADOS version 2. The ADOS is promoted as a generic semi structured, standardized assessment of social interaction, communication, play, and imaginative use of materials for individuals suspected of having ASD [10]. ADOS was first published in 1989 with several updates since then [20].

For the course, the ADOS were advertised as being a ‘semi structured standardized measure of communication, social deficits, and play associated with ASD’. The language of ‘standardized’, ‘measure’, ‘deficits’ depicts an empirically valid and quantifiable approach to identifying a medical problem. There are 5 modules to choose from. Each module rates similar phenomena, but uses some different activities for different intellectual levels. The 5 modules are: Toddler Module, Module 1, Module 2, Module 3, and Module 4. The training course I attended incorporated introductory and general explanatory content, followed by explanation of the activities and method for rating ‘symptoms’ in Module 1, Module 2, Module 3, and Module 4. For each of these 4 modules we then watched a videotaped session of an assessment and had to use the relevant ADOS schedule to make our own ratings before the course facilitator discussed each rating and showed us how to arrive at the ‘correct’ rating Table 1.

Table 1: My scores and the â??correctâ?? scores for the ADOS training videos.

Before presenting my reflections, it’s worth remembering the overall context, which reflects several levels of assumptions. This includes that ASD exists as a discrete, natural ‘thing’, that this ‘thing’ can be identified and quantified, that identification and measurement can be done validly and reliable through specific assessment tools, that ADOS is one such tool, that the items in ADOS encompass and identify the ‘symptoms’ that make up ASD, that ADOS has good psychometrics to enable sensitivity and specificity, and that you can be trained to administer ADOS in a ‘standardised’ way to make ADOS reliable and subject to little inter rater variation. Each assumptive level is open question. If any of these assumptions are unwarranted, then the validity of ADOS is also open to question.

The training

The ADOS assessment involves an interviewer (whom I shall refer to as the ‘examiner’) and a subject (whom I shall refer to as the ‘patient’). The examiner presents a series of tasks for the patient to complete in a limited time and then removes each task at the end of this time and moves on to the next one. The whole process is meant to take a standardized minimum of 40 minutes and maximum of 60 minutes. The behaviour of the patient is observed and given empirical ratings, but should not be allowed to change or modify the examiners approach, including the requirement to produce all the tasks in sequence. Tasks include free play; describing a picture book story; play with miniature figures; completing a puzzle; and, with older patients, questions about their social life; understanding of relationships; and hopes for the future. Throughout the assessment the examiner is looking for the presence of certain ‘symptoms’ or absence of ‘normal’ behaviour, which will then be used to complete the numerical rating system.

The language used was based in assumptions that went unnoticed (or not commented upon if they were) during the course. There was no acknowledgement that these assessments take place in a particular context and setting (such as a medical clinic following concerns expressed by someone about a child) or that the behaviour of the examiner could have an impact on how the patient subsequently behaved. The assumption was that the context and examiner side of the relational dynamic in the assessment room are non-significant, so that what emerges during the assessment is purely the result of the interiority of the patient. Throughout the two days the objectifying language used betrayed this assumptive framework. For example, ADOS can be ‘standardized’ and made ‘objective’, features that achieve a rating are ‘symptoms’, and that the job of a ‘good assessor’ will be to ‘look for symptoms’ and ‘abnormal’ behaviour/expressions.

The quasi autistic rigidity of seeing only ‘real’ internal qualities in the patients was a recurrent theme. Most of the questions I asked during the course stemmed from genuine puzzlement as to how certain patient behaviour in the videos could be ascribed as solely the consequence of symptoms in the patient. This led to circularity in logic where the course facilitator insisted that what was being observed could only be understood as being the manifestations of ASD spilling out into the assessment session, due to the expertise/ experience of the examiner in conducting a ‘standardized’ assessment. Thus, as the facilitator guided us through the scoring, they kept referring to the “fact” that this or that symptom occurred. Non-pathologising interpretations were not tolerated. We were all being trained to become examiners with a keen eye for noticing every minutiae of the ‘not normal’ about the patient.

Thus a mechanistic view of social interaction emerged. The scenarios/tasks are set up as if the examiner, their actions, and the environment, exist as controllable variables so that what emerges irrefutably demonstrates the patients’ social abnormalities. In one example, the course facilitator, discussing a previous patient’s interactions with his mother, demonstrated this one sided view of social interaction once interactions are viewed through the prism of ASD ‘symptoms’. In this example she recounted a story told to her by a parent of a child brought to their clinic. Apparently this mother had told her child, “Why do you never look at me?” The child then started to look at her. The mother now complained, “Why do you stare at me?” The confused boy now decided maybe he should learn to look at her and then away from her. His mother now complained, “Why do you move your eyes from one place to another?” According to this story, the boy eventually developed a complex formula for how long to look at her and away from her. In recounting this tale the facilitator made no comment about the mother’s role in this developing relational discomfort – it was all the result of this child’s (at the time undiagnosed) ASD.

The symptoms and ratings

No allowance was made for the gender or cultural relevance of the activities/questions in the ADOS. The illusion of objectivity starts dissolving when you see the wording of what you are being asked to rate. For example, for rating ‘Stereotyped/idiosyncratic use of words or phrases’ a mark of 2 (indicating high degree of abnormality) is given if ‘often uses stereotyped utterances or odd words or phrases, with some other language’. A mark of 1 (indicating some level of abnormality) is ‘Use of words or phrases tends to be more repetitive than that of most individuals at the same level of expressive language, but not obviously odd.’ For, ‘Quality of social overtures’, 1 is ‘Slightly unusual quality of some social overtures. Overtures may be restricted to personal demands or related to the child’s own interests, but with some attempt to involve the assessor’, 2 is ‘Significant minority (or more) of inappropriate overtures; many overtures lack integration into context and/ or social quality.’ Note that words like ‘often’, ‘unusual’, ‘quality’, ‘some’, ‘significant’, and so on, all require an examiner to interpret they do not lend themselves to establishing objective facts. All the ratings are similarly open to subjective interpretation.

Reliance on the interpretive bias of the examiner was regularly revealed. For example in one video assessment we observed, I saw the child regularly smiling, but facilitator said this was not smiling but the symptom of ‘smirking’. Even if it was ‘smirking’, it is difficult to understand why that should be considered a medical ‘symptom’. Other behaviour rated included: ‘unusual’ use of words; quality of child’s attempt to initiate interaction; whether patient requests things from the examiner; not spontaneously giving toys or other objects back to the examiner; not showing toys or other objects (e.g. by holding them up) to the examiner; lack of flexible, creative use of objects (e.g. a doll) in a representational manner; unusual sensory interests; and so on, all open to interpretive variation. All the ratings are of this nature. They bring up questions of where notions of appropriate/inappropriate, normal/abnormal, healthy/ symptomatic etc. are derived from and on what authority they are based.

My genuine interpretations using the ADOS scoring systems were quite different to that of what the facilitator explained were the correct ‘objective’ scores, particularly for the two younger patients in the videos for Modules 1 and 2. In fact when Module 1 video was shown, I was convinced that this was being shown to illustrate an assessment of a ‘normal’ child to show us the contrast, but it turns out all the videos showed someone who had been given a diagnosis of ASD.

Watching the videos of assessments

I found the videos of Module 1 and 2 assessments painful to watch. The examiner moves quickly from one activity to another, giving each activity a few minutes. In these modules the children were around 3 and 5 years old. The ‘objectification’ of their behaviors had more to do with power and a privileged construction of ‘truth’ than the discovery of anything intrinsic to the child. Both children seemed to me to become uncomfortable and in different ways uncooperative, in my opinion, due to, at least in part, the unusual context and behavior of the examiner a stranger they had just met. We could only comment on and then code the patient’s behavior, but were not allowed to interpret the patient’s possible affect or the relational/contextual nature of interactions. It seemed to me that this ‘test’ of social communication was done through setting up a deliberately provocative environment and expecting these young patients to acquiesce to the examiners unusual demands.

Thus in the videos ADOS looks more like a test of social conformity to a bossy adult’s constantly shifting demands. In one video, early in the session, after the examiner removes the toys the 5 year old patient was enjoying playing with; the patient stands with his back to the examiner and says “You’re not my friend”. The rest of the session plays out a complex interaction, part at times, hilarious rebellion by the young patient, part engagement, and part distressing to watch pre-assuring and cold detachment by the examiner. To me the repetitive questioning, at times exaggerated unnatural smiling and high pitched squeaky childish voice of the examiner, appeared more unusual than the reaction of the child. However, using the ADOS this child had ASD and was apparently the ‘abnormal’ of the two.

In modules 3 and 4, I wondered about the age appropriateness of toys, articles and questions used. I am not sure how I would answer when I was younger (as a 12 and 17 year old in the video assessments had to) questions like “What does a friend mean to you?” “Do you ever think about a long term relationship or getting married?” “Do you take care of your own money?” “Do you have plans or dreams for the future?” According to our rating scales there are normal and pathological ways to answer these questions. Both of these older children spoke about experiences of being bullied. Even this was seen as further evidence that they are incompetent, with a hidden message that the ASD was the reason they were bullied (i.e. their ‘disorder’ caused others to treat them badly). Thus in the assessments there seemed little escape from interpreting whatever happened as a symptom: Whether they engage, how they engage, how they talk, what they say, what they don’t say, how they look, what they do, what they don’t do and so on.

ADOS is not a diagnostic tool

ADOS are a diagnostic trap. An invented assessment, for an invented set of symptoms that is subjective and lacks insight into role of context and inter subjective nature of relationships. It attempts to identify relational ‘deficits’ whilst demonstrating the instruments own lack of awareness about the nature of relationships. It relentlessly seeks to uncover evidence of ‘abnormalities’ and creates a context where the examiner can readily find it. It is a system that catches many in its net, from young kids who won’t do as the examiner instructs and in the way the examiner believes they should, to older ones who have an interesting turn of phrase [20]. It constructs rather than discovers knowledge with the creators, sellers, and now the many examiners who carry out ADOS assessments, believing that they ‘know’ how the universal, culture, gender; sexuality neutral person should and shouldn’t function. It is shamelessly promoted and sold around the world subjecting ever more children and adults to its perverse normalization/pathologisation agenda [21].

Conclusion

ASD is part of the paradigm of medicalising, pathologising, and individualizing that lacks a solid scientific foundation. It is a brand that can be readily monetised through books, programmes, courses, research, and ‘diagnostic’ tools like ADOS. Despite the lack of scientific validity and the failure to disprove the null hypothesis, meaning ASD cannot be considered a medical diagnosis; it has been remarkably resilient for a variety of reasons. There is virtually no journal publication that has seriously critiqued the foundational assumptions that constructed ASD. As a result there is little critical appraisal of crucial elements of practice that underpin practice, such as the ADOS. I hope this article can serve as part of an attempt to start a serious debate on the foundational assumptions underpinning ASD and related ‘diagnostic’ tools.

References

1. Baron-Cohen S, Scott FJ, Allison C (2009) Prevalence of autism-spectrum conditions: UK school-based population study. Br J Psychiatry 194(6): 500-509.
2. Bracken P, Thomas P, Timimi S (2012) Psychiatry beyond the current paradigm. Br J Psychiatry   201(6):430-434.
3. Goldacre B (2012) Bad pharma. London: Fourth Estate. 488 pages.
4. Gøtzsche P (2013) Deadly medicines and organised crime: How big Pharma has corrupted healthcare. London:  Radcliffe Publishing.
5. Whitaker R, Cosgrove L (2015) Psychiatry under the influence: Institutional corruption, social injury, and prescriptions for reform. New York: Palgrave Macmillan.
6. Drury N (2014) Mental health is an abominable mess: Mind and nature is a necessary unity. NZ J Psychol 43(1): 5-17.
7. Timimi S (2014) No more psychiatric labels: Why formal psychiatric diagnostic systems should be abolished. Int J Clin Health Psychol 14(3): 208-215.
8. Viola S, Moncrieff J (2016) Claims for sickness and disability benefits owing to mental disorders in the UK: Trends from 1995 to 2014. Br J Psychiatry Open 2(1): 18-24.
9. Lord C, Risi S, Lambrecht L (2000) The autism diagnostic observation schedule generic: A standard measure of social and communication deficits associated with the spectrum of autism. J Autism Dev Disord 30(3): 205â??223.
10. Kleinman A (2012) Rebalancing academic psychiatry: Why it needs to happen and soon. Br J Psychiatry. 201(6): 421-422.
11. Thomas P, Bracken P, Timimi S (2012) The limits of evidence based medicine in psychiatry. Philos Psychiatry Psychol 19, 295-308.
12. Lotter V (1966) Epidemiology of autistic conditions in young children: I. prevalence. Social Psychiatry 1: 124â??137.
13. Timimi S, McCabe B (2016) What have we learned from the science of autism? In K. Runswick-Cole, R. Mallet, S Timimi (Eds.), Re-thinking autism: Diagnosis, identity and equality (pp. 30-48). London: Jessica-Kingsley.
14. Timimi S, McCabe B, Gardner N (2010) The myth of autism: Medicalising boysâ?? and menâ??s social and emotional competence. Basingstoke: Palgrave MacMillan.
15. Joseph J (2006) The missing gene: Psychiatry, hereditary, and the fruitless search for genes. New York: Algora.
16. Joseph J (2015) The trouble with twin studies: A reassessment of twin research in the social and behavioral sciences. New York: Routledge.
17. Al-Jawahiri R, Milne E (2017) Resources available for autism research in the big data era: A systematic review. PeerJ 12(5): e2880.
18. Waterhouse L, London E, Gillberg C (2016) ASD Validity. Rev J Autism Dev Disord 3: 302â??329.
19. Lord C, Rutter M, Goode S (1989) Autism diagnostic observation schedule: A standardized observation of communicative and social behavior.  J Autism Dev Disord 19(2): 185â??212.
20. Whitaker R (2010) Anatomy of an epidemic. New York: Crown.
21. Runswick-Cole K, Mallet R, Timimi S (Eds.) (2016) Re-thinking autism: Diagnosis, identity and equality. London: Jessica-Kingsley.