DORA AI Capabilities Model

Survey Questions

Responses to the following questions were used in the analysis published in the DORA AI Capabilities Model report.

AI adoption

AI adoption is measured as a factor composed of three highly interrelated variables, reliance, trust, and reflexive use.

Reliance

• In the last 3 months, how much have you relied on AI at work?
  Not at all A little A moderate amount A lot A great deal

Trust

• In the last 3 months, how much did you trust the quality of AI-generated output at work?
  Not at all A little Somewhat A lot A great deal

Reflexive use

• In the last 3 months, when you encountered a problem to solve or a task to complete at work, how frequently did you use AI?
  Never Sometimes About half the time Most of the time Always

AI capabilities

The seven AI capabilities are shown to amplify the benefits of AI adoption.

Clear and communicated AI stance

• Has your organization introduced centralized training practices across teams on how to use AI tools? Please select all that apply
  Yes. Mandatory training Yes. Optional training No training provided
• How frequently does your team have brown bag sessions, peer demos, or informal walkthroughs about the use of AI?
  Never Rarely About once a month About once a week Multiple times a week
• How important of a role does AI have in your company’s strategy?
  Not at all important Slightly important Moderately important Very important Extremely important I don't know
• To what extent do you feel that the use of AI at work is mandatory?
  Not at all Slightly Moderately Very much Extremely
• To what extent does your organization support you with experimenting with AI?
  Not at all Slightly Moderately Very much Extremely
• To what extent does your organization’s AI policy directly apply to your work?
  Not at all Slightly Moderately Very much Extremely
• To what extent is it clear how you're allowed and not allowed to use AI at work?
  Not at all clear Slightly clear Moderately clear Very clear Extremely clear
• To what extent is it clear which AI tools you're allowed and not allowed to use at work?
  Not at all clear Slightly clear Moderately clear Very clear Extremely clear

Healthy data ecosystems

• How easily can you access and analyze the internal data sources you need to complete your work?
  Not at all easily Slightly easily Moderately easily Very easily Extremely easily I don't know or NA
• How often are you unable to use important internal data because the data is siloed?
  Never Sometimes About half the time Most of the time Always I don't know or NA
• How would you rate the overall quality of the data you typically rely on for your work?
  Extremely bad Moderately bad Slightly bad Neither good nor bad Slightly good Moderately good Extremely good I don't know or NA
• If you need a particular data point, how likely is it that you can get a definitive answer within 1 hour of searching?
  Not at all likely Slightly likely Moderately likely Very likely Extremely likely I don't know or NA

AI-accessible internal data

• How often have you inputted or provided internal company information or data (e.g., documents, code, spreadsheets, text) to an AI tool?
  Never Sometimes About half the time Most of the time Always
• Over the last month at work, how frequently have you used AI within your organization to help you retrieve and use information from internal data sources (for example, for answering questions, generating reports, or improving workflows)?
  Never Sometimes About half the time Most of the time Always
• To what extent do the models have access to internal data sources (for example databases or codebases) or applications (for example wikis or work management systems)?
  Never Sometimes About half the time Most of the time Always
• When using an AI tool for your work, how often have its responses or outputs seemed to you to have utilized internal company information or data as context (e.g., by referencing specific internal projects, data, or terminology in its explanations)?
  Never Sometimes About half the time Most of the time Always
• When you have asked an AI tool about internal company matters (for example, specific project details, internal reports), how often did its response lead you to believe it had accessed internal company information or data to generate that response?
  Never Sometimes About half the time Most of the time Always

Strong version control practices

For the primary application or service you work on, which of the following assets are managed in a version control system? (please indicate for each)

• Application code
• Application configurations
• Code for automating build and configuration
• Prompts for AI systems
• System configurations

• Thinking about the most recent change that was committed to the primary application or service you work on, approximately how many lines were changed (added, edited, or removed) as part of that commit?
  1—10 lines were added / edited / or deleted 11—50 lines were added / edited / or deleted 51—100 lines were added / edited / or deleted 101—250 lines were added / edited / or deleted More than 250 lines were added / edited / or deleted I don't know
• When actively changing code or configuration for the primary application or service you work on, approximately how frequently do you commit changes to version control?
  I don't commit changes to version control Rarely About once a week About once a day At least twice a day At least once an hour Every few minutes I don't change code or configuration for the application or service.
• When actively changing code or configuration for the primary application or service you work on, approximately how much do you rely on the ability to undo, revert, or rollback changes?
  Never: I don't have this ability or rarely use it Very Little: I rarely rely on it. It's more of a theoretical safety net or for very major unforeseen issues. Occasionally: I rely on it sometimes typically as a comfort or for clear mistakes but it's not a core part of my active workflow. Heavily: I rely on it a lot. It's integral to my workflow for experimentation; managing complex changes; and quick recovery from missteps. Constantly: I rely on it all the time. It's fundamental to how I make changes; allowing for rapid iteration; risk mitigation; and frequent course correction. I don't change code or configuration for the application or service.

Working in small batches

• For the primary application or service that you work on, approximately how long does it take a developer to complete the work assigned in a single task (for example, card, ticket, or story)?
  Less than one hour Less than one day One day to one week One week to one month More than one month I don't know
• For the primary application or service that you work on, approximately how many changes (for example, pull requests, merge requests, or change lists) are combined into a single release or deployment?
  Only 1 change is included in a typical deployment 2—10 changes are included in a typical deployment 11—50 changes are included in a typical deployment 51—100 changes are included in a typical deployment More than 100 changes are included in a typical deployment I don't know
• Thinking about the most recent change that was committed to the primary application or service you work on, approximately how many lines were changed (added, edited, or removed) as part of that commit?
  1—10 lines were added / edited / or deleted 11—50 lines were added / edited / or deleted 51—100 lines were added / edited / or deleted 101—250 lines were added / edited / or deleted More than 250 lines were added / edited / or deleted I don't know

User-centric focus

Thinking about the primary application or service you work on, to what extent do you agree or disagree with the following statements?

• Creating value for our users is our focus.
• The experience of our users is our top priority.
• We believe that focusing on the user is key to the success of the business.
• We have a clear understanding of what our users want to accomplish with our application or service.
• We leverage user feedback to continuously revisit and reprioritize features.

Quality internal platform

• In the last 3 months, did you use a platform at work?
  I didn't use a platform I used a platform I used multiple platforms I don't know

To what extent does your platform(s) demonstrate the following characteristics?

• The platform behaves in a way I would expect.
• The platform effectively abstracts away the complexity of underlying infrastructure
• The platform gives me clear feedback on the outcome of my tasks.
• The platform helps me build and run reliable applications and services.
• The platform helps me build and run secure applications and services.
• The platform helps me follow required processes (e.g., code reviews, security sign-offs).
• The platform is easy to use.
• The platform provides the tools and information I need to work independently.
• The platform team acts on the feedback I provide.
• The platform's user interface (UI) is straightforward and clean.
• The tasks I perform on the platform are well-automated.

• Does your organization have a dedicated platform team?
  No Yes we have a dedicated team Yes we have multiple dedicated teams I don't know

• Are you on the dedicated platform team?
  No Yes

Outcomes

Outcomes represent the goals of professionals on technology-driven teams. We want to understand how outcomes differ when comparing different levels of AI adoption.

Team performance

How would you rate your team's current performance in the following areas?

• Adapting to change
• Being able to rely on each other
• Delivering innovative solutions
• Effectively collaborating with each other
• Efficiently working together

Code quality

• How would you rate the quality of code underlying the primary service or application you work on?
  Very low quality Low quality Moderate quality High quality Very high quality I don't know or NA

Individual effectiveness

• In the last 3 months, how effectively were you able to perform your tasks and responsibilities at work?
  Not effectively at all Slightly effectively Moderately effectively Very effectively Extremely effectively
• In the last 3 months, how much impact do you think your work has had?
  None at all A little A moderate amount A lot A great deal
• In the last 3 months, how productive did you feel in your work?
  Not at all productive Slightly productive Moderately productive Very productive Extremely productive

Product performance

For the primary service or application that you work on, how would you rate your application or service’s current performance across the following areas?

• Doing what it is supposed to do
• Helping people accomplish what is important to them
• Keeping user information safe
• Performance metrics like latency
• Reliability and availability for users
• Usability and ease of navigation

Reduced friction

• In the last 3 months, to what extent did friction hinder your work? Friction includes things like lack of smoothness, unnecessary manual work, or unexpected obstacles.
  None at all A little A moderate amount A lot A great deal

Software delivery throughput

The next set of questions are about various outcomes of your work. For the primary application or service you work on...

• How long does it generally take to restore service after a change to production or release to users results in degraded service (for example, lead to service impairment or service outage) and subsequently require remediation (for example, requires a hotfix, rollback, fix forward, or patch)?
  More than six months Between one month and six months Between one week and one month Between one day and one week Less than one day Less than one hour I don't know or NA
• How often does your organization deploy code to production or release it to end users?
  Fewer than once per six months Between once per month and once every 6 months Between once per week and once per month Between once per day and once per week Between once per hour and once per day On demand (multiple deploys per day) I don't know or NA
• What is your lead time for changes (i.e., how long does it take to go from code committed to code successfully running in production)?
  More than six months Between one month and six months Between one week and one month Between one day and one week Less than one day Less than one hour I don't know or NA

Organizational performance

For the following statements, we are interested in learning about your organization's performance. For each of the following performance indicators, how did your organization do relative to its goals over the past year? If you are in a contract position, consider your current client to be your organization.

• Achievement of organizational and mission goals
• Customer satisfaction
• Increased number of customers
• Operating efficiency
• Quality of products or services provided
• Relative market share for primary products
• Your organization’s overall performance
• Your organization’s overall profitability