Observation Test Reliably Rated Disruptive Behavior in Kids With ADHD

A structured clinic play session, scored from video by coders who knew nothing about the child, rated restless and angry behavior consistently in 212 Dutch children aged 5 to 12 with ADHD. Its scores matched parent and teacher questionnaires only loosely, and it could not capture inattention.¹

What the DB-DOS Observation Test Is

ADHD is usually diagnosed from interviews and rating scales filled out by parents and teachers. Among developmental conditions, it is unusual in having no standard performance test or structured clinic observation built into routine diagnosis.¹

The Disruptive Behavior Diagnostic Observation Schedule (DB-DOS) is a structured session designed to provoke and record problem behavior in a controlled setting. A child works through set tasks while a camera records from behind a one-way mirror, and trained coders later rate each behavior from 0 (typical for the age) to 3 (clearly atypical).¹

The tasks apply mild pressure on purpose. Parent-led and examiner-led activities are set up to give a child the chance to show noncompliance, frustration, impulsivity, rule-breaking, or anger.¹

The original DB-DOS was built for preschoolers. In its first reliability study, it coded 21 disruptive behaviors in two domains across parent and examiner settings and showed good agreement between coders and across repeat sessions.² A Dutch team later adapted it and tested it in 193 referred preschoolers and 58 typically developing children.³

How Researchers Adapted It for Children Aged 5 to 12

Bierens et al. rebuilt the materials, rewards, instructions, and task demands for school-age children. The roughly 60-minute session ran in 2 settings:

• Parent context: the child does tasks with a parent, the setting where conflict is most familiar
• Examiner context: the child does tasks with an unfamiliar adult

A third setting with both adults present produced too little problem behavior to score and was dropped.¹

Sample: 212 children with a clinical ADHD diagnosis, average age 8.5, about 76% boys, recruited from 2015 to 2021. About 37% also met research criteria for oppositional defiant disorder (ODD; a pattern of angry, defiant behavior), 17% for probable autism, and 34% for emotional problems such as anxiety or low mood.¹

Repeat sessions: the children were enrolled in a randomized trial comparing an elimination diet with a healthy diet. They completed the DB-DOS before treatment, after 5 weeks (206 children), and after 1 year (160 children).¹

Blinding: videos were stripped of anything revealing the treatment or time point. Five coders, unfamiliar with each child, did not know the diagnosis, the diet group, or whether a session came before or after treatment.¹

Coders Agreed on Restless and Angry Behavior, but Not on Inattention

After pruning, 19 behaviors remained. They grouped into 2 dimensions that held up in both settings, at all 3 visits, and in boys, girls, younger, and older children:¹

• Behavioral regulation (9 items): talking excessively, answering impulsively, trouble waiting, interrupting
• Anger modulation (10 items): defiance, noncompliance, aggression, verbal aggression, poorly controlled negative feelings

Interrater reliability measures whether 2 people scoring the same video give the same score. Using an intraclass correlation (ICC), where 1.0 means perfect agreement, values were 0.92 to 0.93 for behavioral regulation and 0.89 to 0.93 for anger modulation in 20 double-coded sessions.¹

Internal consistency asks whether the items in a scale move together. It ranged from 0.66 to 0.81 (Cronbach’s alpha), acceptable to good; the weakest value was behavioral regulation in the parent setting.¹

Inattention dropped out. Most attention items, such as “does not seem to listen” or “difficulty sustaining attention,” were removed because they were hard to provoke and hard to code reliably from a single camera angle without close views of the face.¹

So the test works best for overt, visible behavior. A child with mainly inattentive ADHD and no outbursts or restlessness would score low on it.

Children Showed More Problem Behavior With a Parent Than With an Examiner

At the first session, average scores were higher with a parent than with the examiner on both dimensions:¹

• Restless, impulsive behavior: 9.5 vs. 7.3 out of 27
• Anger and defiance: 5.8 vs. 2.4 out of 30

The parent setting stayed higher at 5 weeks and 1 year. Earlier DB-DOS work in 497 young children also compared parent and examiner settings, and the researchers note the same pattern of contextual variation in that literature.¹⁴

Bierens et al. suggest the 2 settings tell clinicians different things. Familiar parent-child routines bring out everyday conflict. Problem behavior with a stranger is less common, so when it appears it may point to more widespread difficulty with self-control.¹

Descriptively, boys scored higher than girls, and older children showed fewer observable problem behaviors than younger ones.¹

DB-DOS Scores Matched Parent and Teacher Ratings Only Loosely

To check validity, the researchers correlated DB-DOS scores with questionnaires on conduct problems, hyperactivity, executive function (planning and self-control skills), autism traits, and parenting stress. Most correlations were small to moderate, mostly between 0 and 0.35, and were most consistent in the parent setting.¹

A few examples show the scale of the overlap:¹

• Conduct problems: total DB-DOS scores correlated 0.12 to 0.24 with parent- and teacher-rated conduct problems across the 3 visits
• Autism traits: some of the stronger links, up to 0.38 with teacher-rated autism traits at 1 year

The autism link may reflect the format: the DB-DOS is a structured social interaction full of unexpected, frustrating, and disliked demands.¹

Parents, teachers, and a clinic camera each see a child in a different setting, so the researchers expected related but non-identical views.

Repeat Sessions Captured Some Change Over a Year

Average scores fell over the year in both settings. In the parent setting, restless, impulsive behavior went from 9.4 to 7.5 and anger and defiance from 5.6 to 3.8.¹

Scores were also fairly stable from visit to visit (total-score ICC 0.78 from baseline to 1 year). Change on the DB-DOS lined up only modestly with change reported by parents and teachers. Most of those correlations were small and not significant; parent-setting anger scores at 1 year were one exception (r = 0.20).¹

One finding favored the observation. DB-DOS change tracked parent- or teacher-reported change more closely than parent and teacher reports tracked each other. Teacher-reported change at 5 weeks did not predict teacher-reported change at 1 year (r = 0.06).¹

None of this shows the diets worked. Children were in treatment between sessions, and with no untreated group, falling scores could reflect diet effects, normal development, repeat testing, or children whose scores were unusually high at the start drifting back toward average.¹

Limitations of This DB-DOS Study

• Only children with ADHD: there was no typically developing or other clinical comparison group, so the study cannot set cutoffs or show whether the test tells ADHD apart from other conditions.
• Small reliability subsample: coder agreement came from 20 double-coded sessions.
• Treatment between sessions: stability numbers are not pure test-retest reliability, and change cannot be credited to treatment.
• Mostly boys, one country: about 76% boys from a single Dutch diet trial; age and sex patterns were descriptive only.
• Inattention not covered: the final scales capture restless, impulsive, and angry behavior, not attention problems.
• Small inconsistencies in reporting: sample counts differ slightly across the paper (212 in the abstract, 215 in the sample table, 208 in the reliability table), and baseline averages differ by 0.1 to 0.2 points between 2 tables.

What This Means for ADHD Assessment

The researchers are clear that the DB-DOS is not a stand-alone diagnostic test for ADHD, ODD, or conduct disorder. A high score means a child showed a lot of restless or angry behavior under mild pressure, not that the child has a particular diagnosis.¹

• Added view for clinicians: a standardized observation can sit beside parent and teacher reports, which often disagree.
• Blinded outcome for trials: parents and teachers usually know whether a child is on a diet or medication; blind video coding reduces that bias.
• Next steps: norms from typically developing children, tests in ODD or conduct disorder without ADHD, and use in teenagers.