Measuring Creativity and Authenticity in Computer Science

This post is drawn from a soon-to-be presented paper at SIGCSE ’18.

Some curricula and programs weave arts into STEM (and especially computer science) learning, and this combination prompts many to measure “creativity.”  Previous research on creativity offers a model for measuring creativity from four perspectives: product, process, person, and press (place).

The latter two perspectives lend themselves to measurement via survey scale:  Personal creativity is the traits, tendencies, and characteristics of the individual who engages in creative work.  Place is the set of environmental factors that encourage creativity.

We developed a scale to measure these two aspects of creativity within the context of a high school computing course that introduces students to programming through music remixing, EarSketch.  EarSketch is designed to broaden participation in computing by placing students within a thickly authentic learning environment that has personal and real-world relevance in both computing and music.

The two scales, along with their subconstructs, are presented below:

Table 1.  Personal Creativity Scale

Sub-Construct Item
Expressiveness:  conveying one’s personal view through computing ·    I am expressive and creative while doing computing.

·    I can express my unique view of the world through computing.

Exploration: investigating ideas in computing ·    I am capable of exploring many different ideas, options, or outcomes in computing.
Immersion/Flow: feeling absorbed the the computing activity ·    When I do computing, my attention is fully captured by the activity.
Originality: generating unique and (personally) novel ideas in computing ·    I can come up with new ways to do things in computing.

·    I produced something in computing that I never thought was possible.

Sharing: inviting friends/family to see and participate in computing work ·    I want to share what I do in computing with my friends.

·    I want to share what I do in computing with my family.

·    Computing is something I want to do with my friends.

Creative Thinking Skills: approaching computing problems from new perspectives; unorthodox solutions ·    I look for different solutions to a computing problem.

Table 2.  Authenticity Scale

Sub-Construct Item

(Prompt: To what extent did Unit 2…)

Personal: activities that are meaningful to the student ·       Allow me to work on projects that are meaningful to me.

·       Allow me to create things that I am proud of.

Real World: activities reflect aspects of the milieu outside of school ·       Allow me to work on projects that are based in the real world.

·       Help me explore and think about real world issues.

·       Allow me to talk with others about what I learned.

Disciplinary: activities that are authentic to computing ·       Encourage me to think like a computing professional.
Assessment: activities that require students to demonstrate learning ·       Provides me with feedback that helps me solve computing problems.

·       Tells me what I know and what I don’t know about computing.

We administered both scales through a pre/post retrospective student survey at the end of the programming unit.  Personal creativity was administered in both the pre and post condition and authenticity was measured only at post.  We found both scales to be reliable in both the pre- and post- conditions (Personal Creativity: α = .90 and .92; Authenticity: α = .91).  On a survey administered to 205 high school students taking Computer Science Principals, the pre-to-post personal creativity results were significant (p<.001) with a relatively large effect size (d = 1.23).

  • Amabile, T.M. 1990. Within you, without you: The social psychology of creativity and beyond. In Theories of Creativity, A. Runco and R.S. Albert. Sage Publications, Newbury Park, CA, 61-90.
  • Engelman, S., Magerko, B., McKlin, T., Miller, M., Edwards, D., & Freeman, J. (2017). Creativity in Authentic STEAM Education with EarSketch (pp. 183–188). ACM Press. https://doi.org/10.1145/3017680.3017763