Why did you update the norms?

Our general policy at The Learning Bar is to update our norms whenever we modify or update an existing measure, or every five years.  Norms are updated to ensure that the comparison lines appropriately reflect student perceptions as well as capture changing trends in student experiences. We want to ensure that our comparison lines are as timely as possible to facilitate informed decision-making. 

The regeneration process also ensures that measures that may have been created since the release of the previous norms are based on a larger sample size of students.

 

What year are the NSW Gov't norms based on?

The norms are based on data collected from student surveys (primary & secondary) administered in SS1 2023.  The Parent and Teacher norms are based on data collected from SS2 2023.

 

What year are the TTFM National norms based on?

The norms are based on data collected from student surveys (primary & secondary) administered in 2023.

 

Are there norms for the parent and teacher surveys?

Yes, there are NSW Gov’t Norms available within the OCRs, the Trend Reports, and the Interactive Charts.

In 2023, the Parent Survey was updated and the Parent norms were disabled across all reporting for the year. The historical norms are still available for 2022 and prior, and the new norms have been updated to reflect the changes to the Parent survey going forward.

 

How much did the NSW Gov't norm change?

Notable differences: Most of the norms have only changed by 5% or less compared to the previous values. However, norms that have changed by more than 5% from the previous values are listed below. These differences might become more noticeable when the results are broken down by sex and year level. The comparisons mentioned below are based on the aggregate One-Click Report norms.

 

Primary:

• The norm for the NSW Custom measure of Perseverance increased by 6% for students with medium perseverance and decreased by 6% for students with high perseverance. A 5% decrease in Growth Orientation was also noted.
• There was a 5% decrease for Effort and decreases of 8 to 16% for Values Schooling Outcomes, Sense of Belonging, Interest and Motivation. There was a 25% decrease in Positive Homework Behaviour.
• There was a 7% decrease for Bully-Victim.
• For the 10 point measures of Expectations for Success and Relevance there were decreases of 0.6 and 0.7 respectively.
• For the Skills-Challenge measure, there was a 6% increase in students who report having confidence in their skills but low challenge, and an 8% decrease in students who report having confidence in their skills and being appropriately challenged.

Secondary:

• Decreases of 5 to 10% on Aspirations to Finish Year 12, Aspirations to Attend University, Interest and Motivation, Intellectual Engagement Composite, Sense of Belonging, and NSW custom measures: Students with High Optimism, and Students with High Self Concept.
• Decreases of 11 to 16% for Effort, Homework Behaviour, Values Schooling Outcomes, and NSW custom measure Growth Orientation.
• Increases of 5 to 6% were seen for Bully-victim, and NSW custom measures: Students with Low Optimism, and Students with Medium Self Concept.
• For the Skills-Challenge measure, there was a 6% increase in students who report having confidence in their skills but experience low challenge, and an 8% decrease in students who report having confidence in their skills and being appropriately challenged.
• A 0.9 point increase (on a 10 point scale) was seen for Advocacy Outside of School.

Parent:

• All of the norms for the Parent survey have changed by less than 0.5 on a 10 point scale from the historical values. These differences may be more pronounced when results are disaggregated by child year level. Comparisons are made based on the aggregate One-Click Report norms. The most pronounced differences were seen for Learning at Home (0.47 increase) and for Safety (0.46 decrease).

Teacher:

• All of the norms for the Teacher survey have changed by less than 0.2 on a 10-point scale from the previous norm values. 

*Please note: updated NSW Gov’t norms will only appear for those measures selected for inclusion by CESE for each survey window.

 

How much did the TTFM National norm change?

The majority of the norms have changed by 5% or less from the historical values. These differences may be more pronounced when results are disaggregated by sex and year level.

 

A few notable differences at the primary level are:

• Decreases of between 7% to 12% in participation in sports, sense of belonging, homework behaviour, and interest and motivation.
• Increase of 7% in anxiety.
• Norms are now available for advocacy at home and Feel Safe, which were not previously available.

A few notable differences at the secondary level are:

• Decreases between 9% - 11% in sense of belonging, values schooling outcomes, intellectual engagement, interest and motivation, effort, feel safe at school, aspirations to finish high school, and aspirations to attend university.
• Positive homework behaviour decreased by 14%.
• Bullying increased by 7%.
• Norms are now available for self-esteem, hedonia, eudaimonia, goal orientation, and self-regulation, where previously norms had been unavailable.

*Please note: updated TTFM National norms will only appear for those measures selected for inclusion by CESE for each survey window.

 

Why do some norms differ so much in comparison to the previous norms?

There can be many contributing factors that influence a norm, including the amount of time between norm generations, the composition of the sample used for the calculation (e.g., new regions or number of participants), impact of regional or global events (e.g., COVID-19) and other trends that can influence student perceptions (e.g., social media).

 

How do I access the updated norms?

Updated norms will be applied automatically to newly generated One-click and Thematic reports. Within the Interactive Charts the updated norms will be applied when you select the TTFM Line and NSW Gov't Line. Updated norms will also be applied to the Trend Reports.

 

Have the norms in all reports been updated (e.g. OCR text & graphs, ICs, thematic reports)?

The following norms have been updated:

- One-click reports (OCR)

- Interactive charts (IC)

- Replica School (updated but not displayed)

- Trend reports

- Engagement Thematic reports (Student surveys only)

 

Have the norms for all measures been updated?

All measures included in the SS1 2023 Full survey defined by CESE were normed.  In addition, the TTFM national norms within the Interactive Charts were updated for most measures.

 

Does this affect the Replica School in the Interactive Charts too?

Yes, the Replica School Norm for both primary and secondary has also been updated, however, these norms have been disabled due to the removal of the Socio-economic items in the student surveys.

 

Are the new norms displayed retroactively, in reports from previous years?

The new norms will be displayed for all reports (One-clicks, Thematic Reports, Interactives, Trend reports) from this point moving forward.

One-click: The behaviour of the One-click report is to pick up the ‘current year' norm or latest available. When looking at reports from years prior to the 2024 school year when the new norms were applied, the historical norms will be displayed.

Thematics:  The behaviour of the Thematic report is to pick up the ‘current year’ norm or the latest available. When looking at reports from years prior to the 2024 school year when the new norms were applied, the historical norms will be displayed. 

Interactive Charts: When looking at data prior to the 2024 school year, the historical norms will be displayed. For data 2024 and onward the new norms will display.

Replica lines: When looking at data prior to the 2024 school year, the historical norms will be displayed. For data 2024 and onward the new norms will display.

*Special considerations for Year-over-year comparisons: When applying YOY comparisons including 2024 data, the new norm will be displayed. When viewing historical year-over-year comparisons, historic norms will be displayed. This applies to both the purple band and the replica school.

Trend Reports: Trend reports will only display the 2024 norms, even when viewing historical data. The trend reports intend to compare the current data to historical data, therefore only the current norms are relevant.

 

Can I have access to the previous norms again?

The previous norms will still be available when accessing reports prior to the 2024 school year (one-click, thematic reports, and interactive charts). 

Trend reports only display the most recent norm, as the historical data is always shown alongside every snapshot.

 

When will you next update the norms?

Our goal will be to update our norms every 5 years.  The next update will be considered for 2029.  Norms for any new measures or reports will be developed as necessary.

 

Who creates these norms?

The norms are developed by the Research Department using procedures established by Dr. Doug Willms, the President of The Learning Bar.

 

What do I do, or what does this mean, if I have already set my benchmarks using the old norms?

Even though we have updated the norms, it does not mean the historical norms are no longer valid or useful. The new norms can be viewed as a new piece of information and introduced when appropriate
(e.g. when developing new reports, or setting new benchmarks). The new norms should also bring greater alignment and clarity to those comparisons.

 

Why are the norms only based on a single year?

Historically, the norms were based on 3 years of data due to the limited sample available, however, the standard practice is to use data from a single year. The sample size from one full school year is more than sufficient to construct a norm (>50,000 participants is very generous for norming). We want to ensure that data is as timely as possible and reflects current trends, using multiple years of data may mask differences. Using multiple years of data could also result in the overrepresentation of data from certain states, districts, or schools (this is similar to how we ensure that data from only one snapshot is included within the data set).

 

What changes were made to the Engagement Thematic Report?

In 2024 an updated version of the Student Engagement Thematic report was introduced. This new version features updated norms, as well as updated text on the first page. The first page bullet points were updated to feature up-to-date statistics and useful comparisons for users.