game outcomes project


Production / Friday, January 30th, 2015

A couple of weeks ago, at the start of term, Dave Culyba shared with me the Game Outcomes Project, a series of studies on effective teams and best practices.  I wanted to be able to better evaluate my team’s progress through the semester, and these articles provide such detailed metrics on what makes teams successful.  Now that all five parts are out and I’ve read through the whole series, I thought I’d write about what I found interesting as it applies to what I’ll be doing this semester.

Part 1: The Best and the Rest

Most interesting and surprising to me was that the researchers found out that there was no correlation between the production methodology and the outcome of the game project.  In a school setting where we are taught and told to use Scrum and nothing but Scrum, it is interesting to know that this may not be the “best” way to make a game.  In fact, making modifications to Scrum or using a completely different production methodology can be just as valid.  I think finding what fits the team best is the way to go.

Additionally, there was no correlation between team size, game genre, target platforms or financial incentives above individual performance compensation, and success.  This tells me that the same principles can be used to manage teams as small as my own of 6 or as large as a AAA company, no matter what genre or platform, which makes my time as a producer on different teams each as valuable as the next.  This also reinforces the importance of the team above anything else in the game (technology, genre, type of game etc.), which goes to show that people are by far the most valuable resource in making things.

Part 2: Building Effective Teams

I found this section the most useful of the whole article, with the main takeaway being that making an effort to foster team culture is the most valuable way to ensure a team’s success.

It’s tempting to say that the secret sauce is teamwork – but this would be a trivialization of what is actually a far more subtle issue.

The actual secret sauce seems to be a culture that continually and deliberately cultivates and enables good teamwork, gives it all the support it needs to flourish, and carefully and diligently diagnoses it and fixes it when it begins to go astray.

On top of this conclusion, the article provides lists of helpful statements that were evaluated by game developers to measure teamwork and team culture, based on three books on team effectiveness.  Long story short, they spoke the truth!  Here are the points in summary with my notes for this semester, plus the nifty graphs pulled off the Gamasutra page with the statements that were used to evaluate each point.

Leading Teams: Setting the Stage for Great Performances by J. Richard Hackman – Five Enabling Conditions

  1. It must be a REAL TEAM – yup, this is forced at the ETC
  2. Compelling direction – motivating goal: to please the client, to learn things, to make a difference?
  3. Enabling structure – specifying tasks, roles and responsibilities at the beginning of the semester
  4. Supportive context – team trust, have to build this up over the semester
  5. Expert coaching – project advisers

20150130_gameoutcomesproject_hackman

The Five Dysfunctions of a Team: A Leadership Fable by Patrick Lencioni

  1. Absence of trust – we don’t know each other so well at the start of the semester
  2. Fear of conflict – very common, especially in cross-cultural teams at the ETC
  3. Lack of commitment – everybody’s focused on their electives, getting an internship, etc.
  4. Avoidance of accountability – this is why we have scrum, but we may need to check in more with each other too
  5. Inattention to results – need to playtest

20150130_gameoutcomesproject_lencioni

12: The Elements of Great managing (Gallup data from >10 million employee and manager interviews)

  1. Knowing what’s expected – a little unclear, we are all first years
  2. Materials and equipment – ETC tries to provide what we need to do our work, but good to check
  3. I have the opportunity to do what I do best – not true for everyone in a project team so small, we have to fill in roles as needed
  4. Recognition and praise – I need to be more liberal with dishing this out
  5. Someone at work cares about me as a person – I need to be this person as well
  6. Someone at work encourages my development – and I need to be this person, in addition to advisers
  7. My opinions seem to count – something we need to work on as a team, making everyone’s opinions matter and writing down all ideas and feedback from teammates
  8. I have a connection with the mission of the company – unclear, maybe client will bring more meaning and light to this especially after we meet the kids we are helping
  9. My co-workers are committed to doing quality work – this is a team effort, I don’t think I can speak for everyone
  10. I have a best friend at work – wasn’t statistically significant, but I think on such a small team having two friends who are particularly close might be disruptive to the rest of the team
  11. Regular, powerful, insightful feedback – we need to do feedback as a team (some point in Scrum?) in addition to adviser feedback and faculty feedback during Quarters, Halves, Softs, and player feedback during playtests
  12. Opportunities to learn and grow – not overt in a project setting, teammates may rely more on electives for this but i need to make sure their work on the project gives them opportunities to learn and grow too

20150130_gameoutcomesproject_gallup

The best part is, now I can steal lines like “Team members held one another accountable for their actions” and use them in surveys and evaluations to measure my team’s progress.

Part 3: Game Development Factors

My favourite part of this section was that the researchers provided a self-reflection tool based on their study, which will forecast a team’s chance to achieve the project’s goals and ways to improve odds of a positive outcome.  Kind of like a crystal ball for team projects.  They recommend team members taking the survey anonymously and averaging the results to get a good prediction, and I think I’ll try that sometime towards the end of the semester.

Every game is a reflection of the team that made it, and the best way to raise your game is to raise your team.

Although there is always an element of risk involved, the lion’s share of your own destiny remains within your own control.

Part 4: Crunch Makes Games Worse

I’m not going to write about this part focusing on crunch here as it does not strictly pertain to our semester-long project on a team of six, and this blog post has already gone on longer than I had bargained for.  If crunch comes up on a future project I’ll come back to this part of the Game Outcomes Project, but for now, I’ll say the title sums it up.  Crunch makes games worse.

Part 5: What Great Teams Do

The summary is just as useful as the article, outlining the forty highest recommended practices for successful teams.  Overall the Game Outcomes Project is a good read throughout and a great reference for team leaders (and anyone on a team to some extent).  It certainly made me think harder about production practices and what I can do to not only manage a team but evaluate how well my team is doing.

I’ll leave you with the quoted conclusion from the five part series, as the experts have summed it up better than I ever could:

Despite the inevitable risks involved in game development, the clearest result of our study is that the lion’s share of our destiny is in our own hands.  It comes down to a culture that consciously and deliberately fosters, cultivates, and supports effective teamwork.

We spend enormous amounts of effort optimizing our code and our art assets.  There’s no reason we shouldn’t spend just as much effort optimizing our teams, and we hope this study has pointed the way toward some of the tools to help with that process.

Onward to better, happier, more productive teams.