DORA | DORA Research Questions

Survey Questions

Responses to the following questions were used in the analysis published in the 2022 Accelerate State of DevOps Report.

Architecture

Think about how your team and work are structured. For the primary application or service you work on, please rate how strongly you agree or disagree with the following statements.

My team can deploy and release our product or service on demand, independently of other services it depends upon
On my team, we can make large-scale changes to the design of our system without creating significant work for other teams
On my team, we can make large-scale changes to the design of our system without depending on other teams to make changes in their systems
On my team, we perform deployments during normal business hours with negligible downtime
To complete my own work, I don't need to communicate and coordinate with people outside my team
We can do most of our testing on demand, without requiring an integrated test environment

Burnout

Sustainability of work is important, and burnout is an important indicator of this. Please answer a few questions about how your work has affected you. Please rate how strongly you agree or disagree with the following statements:

I am indifferent or cynical about my work.
I feel burned out from my work.
I feel exhausted.
I feel like I am ineffective in my work.
My feelings about work negatively affect my life outside of work.

Cloud Computing

Please rate how strongly you agree or disagree with the following statements in relation to your use of cloud services.

I can dynamically increase or decrease the cloud resources available for the service or product that I primarily support based on demand
I can monitor or control the quantity and/or cost of cloud resources used by the service or product that I primarily support
Once I have access, I can independently provision and configure the cloud resources and capabilities required for my product or service on demand without raising tickets or requiring human interaction
The cloud my product or service runs on serves multiple teams and applications, with compute and infrastructure resources dynamically assigned and re-assigned based on demand

Continuous Delivery

For the primary application or service you work on, please rate how strongly you agree or disagree with each of the following statements.

Fast feedback on the quality and deployability of the system is available to anyone on the team
My team prioritizes keeping the software deployable over working on new features
Our software is in a deployable state throughout its lifecycle
We can deploy our system to production, or to end users, at any time, on demand
When people get feedback that the system is not deployable (such as failing builds or tests), they make fixing these issues their highest priority

Continuous Integration

The next set of questions ask about committing code and building and deploying software at work. For the primary application or service you work on, please rate how strongly you agree or disagree with the following statements.

Automated builds and tests are executed successfully every day
Automated test failures will block a commit's progress through the pipeline
Code commits result in a series of automated tests being run
Code commits result in an automated build of the software

Demographics and Firmographics

Approximately many people are on your immediate team?
Do you identify as a member of an underrepresented group?
Do you identify as a person with a disability? This includes disabilities associated with seeing, hearing, walking, remembering/concentrating, self-care, or communication.
How many employees work at your organization?
How many years of experience do you have in a role similar to your current role?
Location
Start Date
User Language
What is the principal industry of your organization?
What is your gender?
Which most closely describes your work role?

Development or Engineering DevOps or SRE I am a C-level executive I am a consultant coach or trainer I am a student I do not belong to any department Information Security IT Operations or Infrastructure Manager Network Operations Other (please specify) Prefer not to answer Product Management Professional Services Quality Engineering or Assurance Release Engineering Sales Engineering Sales or Marketing User experience or software analysis
With which racial or ethnic groups do you identify? Select all that apply

American Indian or Alaska Native Asian Black or African American Hispanic Latino or Spanish origin Middle Eastern or North African Native Hawaiian or Pacific Islander Other (please specify) Prefer not to respond White

Documentation

The following questions ask about documentation. We define documentation as internal documentation for the services or applications you work on (e.g., manuals, readmes, code comments, etc.). End-users are not the audience for these documents. Please rate how strongly you agree or disagree with the following statements.

I can rely on our technical documentation when I need to use or work with the services or applications I work on
It is easy for me to understand our technical documentation
It is easy to find the right technical document when I need to understand something about the services or applications I work on
Most of the code for the services or applications I work with is documented
Our technical documentation is well organized
Technical documentation is updated as changes are made
The technical documentation correctly reflects the capabilities and requirements of the primary service or application I work on
When there's an incident or problem that needs troubleshooting, I reach for the documentation

Generative Organizational Culture

Think about your organization when you answer the following questions and rate how strongly you agree or disagree with the following statements. In my organization...

Cross-functional collaboration is encouraged and rewarded
Failures are treated primarily as opportunities to improve the system
Information is actively sought
Messengers are not punished when they deliver news of failures or other bad news
New ideas are welcomed
Responsibilities are shared

Infrastructure Platform

Please rate how strongly you agree or disagree with the following statement.

The product or service I primarily work on was originally designed and architected to run in the cloud.

Organizational Performance

For each of the following performance indicators, how well did your organization meet its goals over the past year?

Response options: Performed well above goals Performed above goals Performed slightly above goals Met goals Performed slightly below goals Performed below goals Performed well below goals N/A or I don’t know

Achieving our organizational and mission goals
Customer satisfaction
Increased number of customers
Operating efficiency
Other measures that demonstrate to external parties that your organization achieve intended results
Quality of products or services provided
Quantity of products or services
Relative market share for primary products
Your organization's overall profitability
Your organizations overall performance

Predictions

Making predictions isn't easy, but some people find it helps them better express their perspective. Please indicate how unlikely or likely you think the following events are for the primary service or application you work on.

Leadership will sponsor one or more software security initiatives in the next year
We will be compromised by a security vulnerability in the next year
We will have a complete service/product outage in the next year
We will have a significant performance degradation in the next year
We will have an increased spending budget for new tools in the next year
We will have an outage due to quota or capacity misunderstanding in the next 90 days
We will increase the amount funding for getting new tools in the next year
We will invest more to improve our build systems in the next year
We will invest more to improve our deployment security and monitoring in the next year
We will invest more to reduce risk for dependencies in the next year

Reliability

These next questions ask about reliability, and how you and your team think about it. For the primary application or service you work on, please rate how strongly you agree or disagree with each of the following statements.

I know what its reliability actually was in the most recent period
It has well-defined targets for reliability (such as Service Level Agreements / Service Level Objectives) that are clearly communicated among the team and to customers
My team regularly reviews and revises reliability targets based on evidence
Reliability reviews are performed throughout the development process for all major features on the applications I work on
When we miss our reliability targets, we perform improvement work and/or re-prioritize

How often do your services meet reliability expectations (i.e., SLOs)?

Security

How established are the following practices for the primary application or service you work on?

Build definitions and configurations are defined in text files stored in a version control system
Build metadata (e.g., dependencies, build process, build environment) about an artifact cannot be edited by build services users
Build metadata (e.g., dependencies, build process, build environment) about an artifact documents all dependencies
Build metadata (e.g., dependencies, build process, build environment) about an artifact includes all build parameters
Build metadata (e.g., dependencies, build process, build environment) is available to the people who need it (e.g. via a central database), and is delivered in a format that they accept
Build metadata (e.g., dependencies, build process, build environment) is either generated by the build service, or by a build-metadata generator that reads the build service
Builds are fully defined through the build script and nothing else
Builds are isolated; they cannot interfere with concurrent or subsequent builds
Every change in a revision's history must be individually reviewed and approved by two trusted persons prior to submission
Production releases are built by using a centralized CI/CD system, never on a developer's workstation
Revisions and their change history are preserved indefinitely
The build metadata (e.g., dependencies, build process, build environment) about how an artifact was produced is signed by my build service
When running builds, build steps are prevented from loading any build inputs dynamically (i.e., all required sources and dependencies are fetched upfront)

Please indicate the extent to which you agree with the following statements.

At my company, the software-security protocol is seamlessly built into our development process
At my company, we have a standardized process for addressing software security across projects
I have access to the necessary tooling to execute security tests
My organization has an effective method for addressing security threats
My organization has an effective method for identifying security threats
Our org has processes in place to identify and document all security requirements for the software our organization develops or acquires (including third-party and open source)
Security requirements are reviewed at regular intervals (annually, or sooner if required)
Security review is conducted for all major features on the applications I work on
Security roles are integrated into our software development team
Security tests are run early in the software development process, either by me or by another team
We continuously engage in automated or manual code analysis and testing for all supported releases, in order to identify or confirm the presence of previously undetected vulnerabilities
We have ongoing efforts to monitor information coming from public sources regarding possible vulnerabilities in the software we use and its third-party components

Software Delivery Performance

The next set of questions are about various outcomes of your work. For the primary application or service you work on...

How long does it generally take to restore service when a service incident or a defect that impacts users occurs (e.g., unplanned outage, service impairment)?

More than six months One to six months One week to one month One day to one week Less than one day Less than one hour
How often does your organization deploy code to production or release it to end users?

Fewer than once per six months Between once per month and once every six months Between once per week and once per month Between once per day and once per week Between once per hour and once per day On demand (multiple deploys per day)
What is your lead time for changes (i.e., how long does it take to go from code committed to code successfully running in production)?

More than six months One to six months One week to one month One day to one week Less than one day Less than one hour
What percentage of changes to production or released to users result in degraded service (e.g., lead to service impairment or service outage) and subsequently require remediation (e.g., require a hotfix, rollback, fix forward, patch)?

0-15% 16-30% 31-45% 46-60% 61-75% 76-100%

Team Characteristics

Think about your team (the people around you who report to the same manager or skip-level manager). To what extent do you agree or disagree with the following statements?

Most of the people that were on this team 12 months ago are still on the team today

Trunk-Based Development

We are interested in the development practices and patterns you follow at work. Please rate how strongly you agree or disagree with the following statements.

All developers on my team push code to trunk / main branch at least daily
Branches and forks have very short lifetimes (less than a day) before being merged to the main branch
There are fewer than three active branches on the application's code repo

Version Control