Measuring AI’s Impact on Software Delivery Speed
James Shore proposes a randomized within-subject repeated-measures trial for comparing software tasks completed with and without AI. The method calls for recording estimates before assignment, clearly defining and practicing both approaches, applying equivalent workflow improvements to both conditions, interleaving assignments, and attributing rework to the original task. The article offers a measurement methodology; it does not provide independent evidence that AI accelerates delivery.
The proposed design seeks to isolate AI’s effect on delivery speed by comparing the same teams across interleaved AI and non-AI tasks, controlling for estimate bias, task differences, and concurrent workflow improvements.
This source provides an excerpt. Continue reading on the original site.