The pattern is not surprising. Claude is highly influenced by the existing codebase. If the surface area is large, inconsistent, or structurally weak, the model will amplify those characteristics. Conversely, if the surface area is small, cohesive and internally consistent, the model’s output improves markedly.

For clean code, three principles dominate.

Strong Domain Models
The core concepts of the system should be explicit, named, and represented directly in code. Behaviour should live with the concepts it belongs to. When the model is coherent, both humans and AI can extend it predictably. When the domain is implicit or smeared across utilities and controllers, every change becomes guesswork.

Encapsulation
Keep data and behaviour together and expose only meaningful operations that reflect the language of the domain. Do not provide mutators (setters) and avoid casual accessors (getters); every accessor that leaks state invites distributed behaviour, breaking the domain model. AI systems are particularly sensitive to porous boundaries, because they will happily reimplement logic.

Minimal Conditional Logic
All but the briefest branching structures are a code smell. They signal decisions being made too late, missing domain concepts, missing polymorphism, or collapsed responsibilities. Nested conditionals increase cognitive load and expand the probability space the model must reason over. Move decisions back to the caller, which already knows the intent, or replace conditionals with richer, polymorphic domain types. This reduces complexity at the point of execution and keeps behaviour aligned with explicit intent rather than inferred state.

When Claude got everything right

That said, it would be misleading to suggest the experience is uniformly negative. Recently I built a small utility application for testing single sign on in a single prompt, and Claude produced exactly what I wanted. The prompt was:

Create a very simple web application for testing authentication with azure.
It should let me sign in, and when I click a button, makes a request to a 
backend service using the bearer token. The backend service is 
../azure-ad-jwt-debugger. Use the msal library as appropriate.

And Claude delivered precisely that. Below is the generated interface…

It signs in with MSAL, retrieves the token, and calls the backend with the bearer token exactly as requested. Interestingly, it did not just implement the minimum described in the prompt. It also added:

• A sign out button
• A structured JSON response display panel

Neither of these were explicitly requested; both were appreciated. You can view the source code here. I have not reviewed it, since in this case, I do not envisage needing to modify it.

In conclusion, when the intent is crisp and the domain small, vibe coding may work extremely well. When the system grows and architectural trade-offs become material, discipline becomes essential. That is not an argument against generative AI. It is an argument for treating it as a powerful amplifier. Code quality matters more, not less, in the age of generative AI. Until models evolve in ways that reason more structurally about long-term design consequences, sustainable AI-assisted development depends on disciplined architecture. If the underlying design is coherent, it accelerates you. If it is messy, it accelerates the mess.

This post explores why experiences with Generative AI assisted software development vary so dramatically. While some developers report order of magnitude productivity gains, others encounter architectural drift, excessive code, and serious operational risk. This gap is not primarily caused by the tools themselves, but by differences in goals, constraints, and methods of use.

Treating Generative AI as a meta tool that operates on intent rather than code mechanics, the post makes explicit its own optimisation goals: negligible operational debt and highly malleable, clean code. A controlled experiment using a small but realistic URL shortener service tests three ways of working with Claude Code.

True vibe coding, where you “forget the code even exists”, proved unreliable. A guided approach using techincal prompts, explicit implementation notes, rules, and manual review produced a clean, low debt system in just over an hour, delivering an estimated eight to sixteen times improvement over manual development. However, these gains should be interpreted cautiously, as coding represents only a fraction of a software engineer’s working time. In contrast, an unattended approach without guidance completed faster but generated substantially more code, significant technical and operational debt, and multiple design regressions.

The conclusion is that Generative AI can deliver dramatic gains, but only when projects are already established or properly bootstrapped, and the agent is strongly constrained. Left unattended, it reliably drifts towards verbosity and accidental complexity. The real question for organisations is therefore not whether to adopt Generative AI, but how to use it in a way that aligns with their long term engineering goals.

Introduction

Jason Gorman’s recent post on the future of software development is causing quite a stir. The comments on the accompanying Hacker News discussion span an extraordinary range of experience with vibe coding, particularly when using Claude Code. Some describe dramatic productivity gains and rapid delivery of complex systems. Others report dangerously misleading output, architectural drift, and large amounts of unusable code. Both sides speak with confidence, often dismissing the other as naïve, reckless, or simply doing it wrong. The discussion reflects a broader pattern which has been bothering me for some time.

I have been a professional software engineer for thirty years, and strong disagreements are nothing new. Functional versus object oriented programming, static versus dynamic typing, competing approaches to testing. What is different this time is the nature of the tool itself. Generative AI does not primarily operate at the level of a language, framework, or architectural paradigm. It operates at the level of intent. You describe the outcome you want and it produces an implementation that attempts to achieve it. For straightforward applications, where the architectural trade offs are limited or inconsequential, this can be highly effective. However, once those trade offs become material, around performance, evolvability, correctness or operational risk, the model is still making implicit architectural decisions without properly accounting for their long term impact. It is improving quickly, but it is not yet a replacement for professional judgement.

It does not matter that Generative AI is unusually non-deterministic. For it to be useful to anyone, it still has to produce desired outcomes with sufficient consistency. That makes the scale and intensity of the reported disagreement surprising. Vastly different experiences are unlikely to be explained by an inherent defect in the tool itself. Allowing for bias and differing personality temperaments does not adequately explain the size of gap this time. The same tool is being described, with equal confidence, as producing dangerous, unmaintainable AI slop on the one hand, and delivering twenty times productivity on the other.

This gap is pushing organisations to make extreme decisions. Some, driven by unrealistic expectations, are rushing too quickly into AI adoption, creating unnecessary anxiety and disruption. Others are avoiding it entirely, missing out on potential benefits and frustrating their internal AI champions. The gap needs to be understood and closed.

What I Want From Generative AI

One possible explanation for difference in experience is that people want different outcomes. If that is the case, then disagreement about Generative AI performance is inevitable. Before comparing tools or techniques, the goals themselves need to be explicit. I want Generative AI to rapidly create applications with the following characteristics:

1. Negligible Operational Debt

Operational debt is distinct from technical debt. Technical debt only incurs a cost when change is required and is often an explicit trade-off. Operational debt creates ongoing risk and unplanned work and, in the worst cases, can consume an entire team’s capacity through incidents and urgent remediation.

2. Unparalleled Malleability

Clean Code is malleable. When we write clean code, we make a comparatively small sacrifice now to reserve the ability to change rapidly in the future. The best way to achieve clean code is to write as little code as possible. This is done by creating a good domain model. When the domain model is good, the code vanishes. As Linus Torvalds is reported to have said:

“Bad programmers worry about the code. Good programmers worry about data structures and their relationships.”

A good domain model requires good encapsulation. Behaviour is contained in one place, colocated with the associated data. Accessors are few, mutators fewer still. Conditional logic is pushed to the boundaries of the application and eliminated internally through polymorphism. The code clearly expresses intent.

I have a nagging suspicion that my attachment to clean code may be what Scott Adams calls Loserthink, and no longer needed in the world of Generative AI. Until that suspicion is proven, I am sticking with it though. The risk of a future filled with vast quantities of even less malleable code than we already have is too great.

My Experience Of Vibe Coding (So Far)

The Good

It is often stated that Generative AI is effective at mechanical or rote tasks, or comparable to the output of a junior engineer, but this is not where it offers the most value. The area where I have found my Generative AI tool of choice, Claude Code, to be most effective is where I have strong high-level judgement about what needs to be achieved, but lack the detailed, low-level knowledge to implement it without time, research, and mistakes. In those cases, the limiting factor is not judgement, but execution.

CI/CD pipelines leveraging Docker provide a good example. I know what I want a pipeline to do, how the stages should fit together, and what correct behaviour looks like. Implementing that by hand usually involves reading documentation, iterating on syntax, and discovering edge cases through failure. Claude is much faster at cycling through that execution loop than I am, especially after installing the GitHub CLI, which is an absolute game changer for working with GitHub Actions!

In summary, Generative AI adds the most value when it operates below my level of judgement but above my knowledge or ability to acquire it. It accelerates low-level execution without being asked to make high-level decisions it is poorly suited to make.

The Bad

At the same time, Claude, has ignored explicit instructions, made false assumptions, implemented changes that were not requested, and drifted away from the intended structure, particularly early in a codebase when there is less existing context. It has also misdiagnosed problems and disappeared down rabbit holes without ever fixing them, or worse, unilaterally decided that they can be safely ignored.

What stands out is how sensitive the results are to relatively small changes in how the tool is used. Claude is not a compiler. The results are not deterministic. Small differences in context, ordering, or phrasing can lead to materially different outcomes, even when the intent appears unchanged. Overall, the outcomes are still positive, but I have yet to achieve the 20x improvement reported by some. Either those reports are grossly overstated, or those making them have found ways to circumvent these issues and make Claude perform consistently well. If the latter is true, I want to learn and adopt their methods, but what are they?

What We Need Is An Experiment

To move this discussion forward, we need something more concrete than confident anecdote. When the same tool is reported to produce both dangerous, unmaintainable systems and dramatic productivity gains, opinion alone cannot tell us whether the difference lies in the tool itself, the goals being optimised for, or the way it is being used. The only way to separate those factors is to make the goals explicit and then test, in a controlled way, whether a particular method of using Generative AI can reliably produce outcomes aligned with them. What is needed is a repeatable test case against which different methods can be evaluated. It must be small enough to run repeatedly, representative of real world software, and sufficiently complex to expose meaningful trade offs and failure modes. A URL shortening service meets those criteria. Here are the stories:

Hypothesis

The vast variation of experience comes from how Generative AI is being used, not from a fundamental weakness in the tool itself.

Apparatus

• Machine: MacBook M1 Pro with 32GB RAM
• Operating system: macOS 15.6.1 (24G90)
• Model: Claude Sonnet 4.5 via AWS Bedrock
• Context window: 1MB
• Claude version: 2.0.76
• Editor: zed (barely used)
• Generative AI execution: Separate terminal window running in plan mode, with safe GitHub and Bash commands pre-allowed
• CLI tooling: GitHub CLI and other macos-tools
• Claude Marketplace: cressie176-claude-marketplace
• Node.js Templates: node-templates
• Stories: shorty/issues

Method 1: Prompt Bootstrapping, Implementation Notes and Manual Accepts (Abandoned)

The initial approach was to use pre-written stories, marketplace skills and interactive prompts to fully implement the URL Shortening service. The intention was to encode structure, constraints, and best practices entirely through instructions. Unfortunately, this proved unreliable, particularly while the codebase was in infancy. Even when instructions were repeated and made increasingly explicit, Claude would occasionally ignore them or drift away from the intended structure. Continuing in this direction wasted both time and tokens. I needed a way to bootstrap the application without Claude.

Method 2: Template Bootstrapping, Implementation Notes and Manual Accepts

For my second attempt, I still worked from pre-written stories, marketplace skills and interactive prompts, but my infrastructure story instructed Claude to bootstrap the service from a custom GitHub template repository. A base service template establishes the core structure, with additional templates layered on top for concerns such as PostgreSQL or other infrastructure. This allows common practices to be shared while still supporting different combinations.

Traditional automation struggles here. As the number of layers increases, reliably merging templates becomes difficult, particularly where cross-cutting concerns are involved. This is where Generative AI proved useful. Each layer includes a “Wiring.md” file describing how it should be integrated into the base. Claude can read and apply these instructions in a way that would be awkward to achieve with scripts. The bootstrap process also made it possible to provide Claude.md files in the templates, but this introduced merge problems as templates were combined. Using rules proved more effective, as each template can include its own rules separately.

Even with templates and rules in place, Generative AI still required technical direction at the story level. Each story therefore includes explicit Implementation Notes recommending an approach, along with reminders to review the relevant rules and skills. These could have been entered interactively, but this would have made the experiment less repeatable and less transparent. Templates capture structural decisions, rules capture behavioural constraints, and stories capture local trade-offs and more nuanced technical decisions. Together, they allow experience to be shared through the artefact itself, rather than through constant explanation or review.

Claude was instructed to install the required templates before implementing any functional stories. With that foundation in place, apart from the occasional and minor course correction, a single prompt was all that was required.

╭─── Claude Code v2.0.76 ────────────────────────────────────────────────────────────────────────────╮
│                                                    │ Tips for getting started                      │
│                    Welcome back!                   │ Run /init to create a CLAUDE.md file          │
│                                                    │ ──────────────────────────────────────────────│
│                    * ▗ ▗   ▖ ▖ *                   │ Recent activity                               │
│                   *             *                  │ No recent activity                            │
│                    *   ▘▘ ▝▝   *                   │                                               │
│                                                    │                                               │
│ arn:aws:bedrock:eu-west-1:808… · API Usage Billing │                                               │
│          ~/Development/cressie176/shorty           │                                               │
╰────────────────────────────────────────────────────────────────────────────────────────────────────╯

  /model to try Opus 4.5. Note: you may need to request access from your cloud provider

──────────────────────────────────────────────────────────────────────────────────────────────────────
> Implement the URL Shortener epic https://github.com/cressie176/shorty/issues/1 one story at a time 
──────────────────────────────────────────────────────────────────────────────────────────────────────
  ? for shortcuts

One deliberate adjustment was made around test-driven development. I allowed Claude to generate tests and production code for a story in a single pass, rather than enforcing a strict red, green, refactor cycle. My assumption was that the model does not benefit from incremental test feedback in the same way a human does, although this remains an open question.

Almost all of my interaction during this phase took place using Claude via the terminal. I reviewed diffs and observed Claude’s workflow there, intervening only when necessary. I didn’t avoid editing in an IDE, I just never felt the need. I only switched to zed once Claude had completed each story, in order to review the changes as a coherent logical unit and to scan through the tests.

I reran the experiment multiple times from the same starting point and received approximately similar results.

Results

The code can be found here: shorty

Tool Rejections (deduped)

1. Don’t duplicate config in tests
2. Prefer single-line if statements
3. Don’t use try-catch for testing errors, use throws/rejects
4. Pass the full redirect config not just the key
5. Use object parameters in constructors
6. Don’t be lazy with assertions (use eq not ok/match when you know the full string)
7. Inject the full error message into the JSX template
8. Suppress expected error logs in tests
9. Destructure { rows } instead of result.rows

Using this approach, Claude correctly implemented the URL shortener service in one hour and seven minutes, with minimal intervention or further prompting. The architectural drift and disobedience seen earlier largely disappeared once the environment was properly bootstrapped.

There were some minor style problems. Claude is overly fond of blank lines, and despite it being mentioned in the implementation plan, it missed that an object’s toJSON() method will be called automatically by Hono’s JSON serialiser. Slightly more serious was that Claude omitted running database migrations from each integration test. The tests still passed because migrations were run from the template Postgres.test.ts, but this may not have run first, potentially causing intermittent failures. I have since updated the node-pg template to run migrations automatically from within the Postgres class to avoid this in future. Overall, these are minor niggles, and the code satisfied my goals of minimal operational debt and cleanliness.

I estimate it would have taken me 1 to 2 working days to produce an equivalent codebase from the same templates without AI. This would suggest that Claude achieved an 8x to 16x improvement. However, the implementation time does not reflect the full cost. These results required repeated iteration on both the stories and the skills. Significant effort went into refining story structure, clarifying implementation notes, and adjusting skills so that Claude behaved consistently.

Method 3: Templated Bootstraping, No Implementation Notes and Automatic Accepts

As another experiment, I tried a deliberately unattended approach without the benefit of implementation notes. The code can be found here: claude-unattended

╭─── Claude Code v2.0.76 ────────────────────────────────────────────────────────────────────────────╮
│                                                    │ Tips for getting started                      │
│                    Welcome back!                   │ Run /init to create a CLAUDE.md file          │
│                                                    │ ──────────────────────────────────────────────│
│                    * ▗ ▗   ▖ ▖ *                   │ Recent activity                               │
│                   *             *                  │ No recent activity                            │
│                    *   ▘▘ ▝▝   *                   │                                               │
│                                                    │                                               │
│ arn:aws:bedrock:eu-west-1:808… · API Usage Billing │                                               │
│          ~/Development/cressie176/shorty           │                                               │
╰────────────────────────────────────────────────────────────────────────────────────────────────────╯

  /model to try Opus 4.5. Note: you may need to request access from your cloud provider

──────────────────────────────────────────────────────────────────────────────────────────────────────
> I want you to implement https://github.com/cressie176/shorty/issues/1 story by story.
  I want you to completely ignore the implementation notes within the GitHub issues for everyting
  EXCEPT the Project Initialisation story. You MUST NOT even read them otherwise they will influence 
  you. To avoid reading them pipe the output from the gh issue command into something to remove 
  everything after the "Implementation Notes" title so that it is completely unavailable to you. 
  When you think a story is done, build, lint, test and commit.
──────────────────────────────────────────────────────────────────────────────────────────────────────
  ? for shortcuts

Results

Claude completed all eight stories in 30 minutes and 34 seconds. On the surface, this looks impressive. In reality the code was a hot mess.

The guided version produced 158 lines of code with no comments. The unattended version produced 271 lines with seven (pointless) comments. That’s 72% more code and 104% more TypeScript for the same behaviour!

Furthermore, the unattended implementation introduced substantial technical and operational debt:

• Unnecesary new services for key generation and expiry management.
• Redirects retrieved from the API were not expired.
• Consecutive database queries were performed using separate client calls, so they were not part of the same transaction, in disregard of the Unit of Work pattern.
• Errors polluted the logs during test runs in disregard of the Suppress Expected Error in Logs guidance.
• It implemented an explicit rude word filter rather than removing vowels, increasing maintenance burden.
• setInterval was used for the maintenance tasks without unref(), which can prevent the process from exiting and does not scale horizontally.
• Functions were far larger, with SQL and JSX inlined making the code harder to follow, in disregard of the Function Design guidance.
• PostgreSQL was used poorly, with application code compensating for weak queries, e.g.
  • Expiry logic was checked in application code rather than expressed directly in a query.
  • Duplicate urls were inserted rather than being upserted.
  • Scheduling was done in application code in disgregard of the pg_cron guidance.
• The codebase was littered with low value comments that narrated rather than clarified, in disregard of the Comments guidance.
• Tests used Monkey patching instead of dependency injection to fake behaviour.
• Fetch was used in tests directly increasing duplication, reducing readability, making them brittle. and in disregard of the Test Client pattern.

Conclusion

Method 1 (Prompt Bootstrapping, Implementation Notes, and Manual Accepts) showed that Claude is ineffective when asked to bootstrap a non-trivial system from scratch using prompts alone. Even with detailed stories and marketplace skills, progress was slow and fragile, with frequent drift making the process an uphill and unnecessary battle. The results suggest that Claude is far more productive when working on an existing or bootstrapped codebase, and that it appears to weight existing artefacts more heavily than abstract guidance from prompts or skills. This limitation motivated the move to a templated bootstrap approach.

Method 2 (Template Bootstrapping, Implementation Notes, and Manual Accepts) demonstrates that Claude can produce high-quality code far more rapidly than even an experienced software engineer, but only when projects are already established or properly bootstrapped, and the agent is guided by effective prompts, whether through marketplace skils, embedded directly in user stories or provided interactively. While marketplace skills represent a long-term investment that can be reused, stories do not. Incorporating detailed implementation notes into each story was necessary for this experiment, but it also made those stories brittle. Real world success will therefore depend either on becoming very good at writing implementation notes up front, or on the engineers driving Claude being skilled enough to provide sufficiently strong just-in-time guidance. Where that is the case, stories that previously took days can be delivered in hours, shifting the primary bottleneck from implementation to story writing. It may be that AI-empowered teams will need to be significantly smaller, narrower in focus, and able to move rapidly between applications and domains to circumvent this bottleneck.

It is also important to put these productivity gains into context. Software engineers do not spend anything close to 100% of their time writing code. Design discussions, team ceremonies, research, troubleshooting, supporting others, administrative work, and training all consume significant portions of a typical working day. Even a dramatic improvement in coding effectiveness therefore does not translate directly into an equivalent improvement in overall productivity.

In contrast, Method 3 (Template Bootstrapping, No Implementation Notes, and Automatic Accepts) demonstrates that Claude is not yet something that can be left to operate unattended while still producing consistently good outcomes. Without strong constraints, it reliably drifts towards verbosity, duplication, and accidental complexity, even when the resulting system is functionally correct. That gap between apparent success and long-term maintainability likely explains much of the current scepticism and pushback.

After all of this, I am still left pondering the following open questions:

1. Were my goals the right ones (particularly my need for Clean Code)?
2. Was my method the most effective way to use Generative AI, and specifically Claude Code?
3. Do other goals matter more in different contexts?

If you are getting better results from vibe coding, what are you optimising for, and how does your approach support that? If you are getting poor results how does your approach differ from mine? I’d love to know.

I have been contributing to open source for almost two decades. It has taught me a great deal about real-world problem-solving, communication, and collaboration. Furthermore, reviewing a candidate’s contributions - whether personal projects, pull requests, or issues, can reveal insights into their creativity, communication skills, and technical mindset. This applies equally to junior developers because even small projects or issues can demonstrate qualities like passion, curiosity, intelligence, diligence, empathy, humility, and respect.

That said, I agree with James that open source contributions should not be a requirement. Plenty of excellent developers do not engage in open source for various reasons. But if someone has made meaningful contributions, I want to hear about them. For me, it is less about the green squares and more about the stories behind them; what interests they have, what challenges they faced, and what they learned along the way.

Open source is just one piece of the puzzle, but it is often a fascinating one. I believe it can be a valuable part of the hiring conversation, no matter what the candidate’s level of experience.

A wrong hire can delay work, frustrate teams, and cost both significant time and money. It is not good for the candidate either. Being in the wrong role causes anxiety, damages confidence, and leaves a blemish on their career history. Furthermore, dealing with someone who underperforms is disruptive, demoralising and exhausting for their manager, their colleagues, and the person themselves. Conversely, a good hiring process means that successful candidates get to work on interesting projects with great colleagues and vice versa. This is why it is in everyone’s interest that the process is effective.

However, the methods used in hiring have faced increasing scrutiny, particularly coding tests, whether take-home or live. While these methods are not perfect, dismissing them outright risks missing valuable insights. Let me explore some of these debates and how thoughtful hiring processes can strike a balance.

Common Criticisms of Coding Tests

Common criticisms of coding tests include:

• Anxiety: Many candidates experience stress caused by live coding exercises, which can lead to underperformance during interviews.

• Time constraints: Take-home projects can be burdensome, especially for those with limited time due to family or other commitments. An ex-colleague even refused to interview for companies who set take-home tests.

• Unrealistic tests: Many coding tests focus on niche skills like recursion or bit masking that are rarely used in day-to-day work. These tests often fail to measure a candidate’s ability to perform in real-world scenarios, leaving a gap in assessing their practical effectiveness.

Why a Live Coding Test Is Valuable

One of the most revealing parts of a coding interview is observing candidates using their toolchain (computer, IDE, command line, etc). An experienced software engineer develops a kind of muscle memory, navigating their environment with ease and efficiency. Whether it is memorised shortcuts, efficient workflows, favourite packages, or thoughtful organisation, these details strongly correlate with hands-on experience. They demonstrate not only a candidate’s technical skill but also their approach to problem-solving and familiarity with their craft. It is not about perfection; it is about the habits and fluency that only come from practice.

A live coding test allows me to observe these behaviours in action. It shows how candidates approach a problem, how they structure their code, and how they troubleshoot unexpected issues. It also provides insight into their communication skills and how they handle feedback, both of which are critical in collaborative environments.

In some cases, it can reveal red flags about a candidate. For instance, I have encountered several candidates who ignored feedback, which raised concerns about their adaptability and willingness to engage constructively. In another example, I asked a candidate to adapt their in-memory solution to one using persistence with a datastore of their choice. To my surprise, they responded with exasperation, raising serious concerns about their suitability for a collaborative, team-oriented culture.

Coding Tests: A Flexible Approach

My preferred approach addresses the concerns around live and take-home tests by offering flexibility. I use tests that do not require company domain knowledge but do demonstrate frequently used software engineering skills. These tests are tailored to the experience level of the role, ensuring they are both relevant and fair. Candidates are encouraged to use whatever resources they have available, including Google, Stack Overflow, and AI tools. I always attempt the test myself to ensure its appropriateness and feasibility. Additionally, at least one existing employee of the appropriate level is asked to attempt it as well. I also try to put the candidate at ease prior to starting the live aspect of the test, rather than jumping straight in and advising them that it is not necessary to finish.

I provide candidates with the coding test beforehand. At a minimum, I ask them to read the exercise and come prepared with their environment ready to go. If they prefer, they can complete the test in advance and walk through their solution during the interview, although I will ask them to make changes so I can observe them work. This allows:

• Time-poor candidates to do the minimum and focus on live problem-solving,

• Candidates who feel anxious to prepare more thoroughly in advance.

This approach gives candidates control over how they prepare and ensures I can still evaluate problem-solving skills, adaptability, and hands-on coding ability. Additionally, I have offered a second chance to candidates who suffered badly due to nerves but who performed well otherwise.

Final Thoughts

To paraphrase Warren Buffett, at the heart of any good hiring process is the search for intelligence, energy, and integrity. Depending on the level of role, knowledge and wisdom become increasingly important too. No hiring process will ever be perfect at evaluating these criteria, but thoughtful adjustments, like offering flexible coding tests, observing how candidates use their tools can make the process more effective. The goal is to use these tools not as rigid gatekeepers but as opportunities to learn about candidates and identify those who will thrive in your team.

Recommended Reading

• Why Can’t Programmers.. Program?
• Interviewing for Evidence

The Problem: Managing Shared Reference Data

Most applications rely on reference data—information that changes infrequently but must remain consistent across the system. However, managing this data across distributed systems introduces challenges:

Meet Filby: Version Control for Reference Data

Think of Filby as version control for reference data. Just as source control systems like Git track changes to source code, Filby manages reference data changes. Like checking out a commit, your applications can use Filby to retrieve reference data for a given change set id. They can also inspect the changelog to find which change set was in effect at a given point in time, and subscribe to reference data update notifications.

Key Features and Benefits

Filby offers a structured approach to managing shared, temporal reference data, delivering several advantages:

1. Safe, Predictable Updates: Deploy new reference data ahead of time and activate it when needed.
2. Consistency Across Systems: The change set mechanism ensures consistency, even with distributed systems.
3. Historic and Future Data Access: Retrieve past data or schedule future changes.
4. Customisable Projections: Tailor views of reference data for specific clients or systems.
5. Version Control: Support backward-incompatible changes with versioned projections.
6. Caching Made Easy: Change sets should never change making them highly cacheable.
7. Local Development Support: Test applications locally using HTTP mocking libraries.

Filby’s Core Concepts

To understand how Filby works, let’s break down its main components:

• Projections: Versioned views of reference data, tailored to specific use cases.
• Entities: The individual pieces of reference data.
• Data Frames: Snapshots of entities at specific points in time, grouped by change sets.
• Change Sets: Logical bundles of updates with a common effective date.
• Notifications: Update events for maintaining synced data across systems.
• Hooks: Custom event handlers triggered by data changes.

┌───────────────┐
│               │
│               │ announces changes via
│  Projection   │────────────────────────┐
│               │                        │
│               │                        │
└───────────────┘                        │
        │ depends on                     │
        │                                │
        │                                │
        │                                │
       ╱│╲                              ╱│╲
┌───────────────┐                ┌───────────────┐                 ┌──────────────┐
│               │                │               │                 │              │
│               │                │               │╲ delivered via  │              │
│    Entity     │                │ Notification  │─────────────────│     Hook     │
│               │                │               │╱                │              │
│               │                │               │                 │              │
└───────────────┘                └───────────────┘                 └──────────────┘
        │ aggregates                    ╲│╱ is raised by
        │                                │
        │                                │
        │                                │
       ╱│╲                               │
┌───────────────┐                ┌───────────────┐
│               │                │               │
│               │╲ is grouped by │               │
│  Data Frame   │────────────────│  Change Set   │
│               │╱               │               │
│               │                │               │
└───────────────┘                └───────────────┘

A Real-World Use Case

Imagine a system managing holiday park data. With Filby:

1. Define entities like “Park” and “Season” in JSON or YAML files.
2. Create a changelog for the Park and Season data updates in JSON, YAML, CSV or SQL.
3. Tailor projections to specific needs, such as a Mobile App requiring a compact view of park details.
4. Use Filby’s API to retrieve park data at any point in time.
5. Add “hooks” to notify other systems that new data and/or projects are available.

               Change
               Hook       Invalidate Cache
           ┌─────────┐   ┌──────────────┐
           │         │   │              │
           │         ▼   │              ▼
┌────┬───────────┬──────────┐       ┌────────┐   GET /changelog/parks/v1        ┌──────────┐
│    │           │          │◀──────│        │◀─────────────────────────────────│          │
│    │           │ RESTful  │       │        │                                  │  Mobile  │
│ DB │   Filby   │   API    │       │  CDN   │                                  │   App    │
│    │           │          │       │        │   GET /projection/parks/v1?id=29 │          │
│    │           │          │◀──────│        │◀─────────────────────────────────│          │
└────┴───────────┴──────────┘       └────────┘                                  └──────────┘
           ▲
           │
           │
┌─────────────────────────────────────────────┐
│                                             │
│               Reference Data                │
│                 Change Sets                 │
│                                             │
└─────────────────────────────────────────────┘
     ▲           ▲           ▲           ▲
     │           │           │           │
     │           │           │           │
┌─────────┐ ┌─────────┐ ┌─────────┐ ┌─────────┐
│         │ │         │ │         │ │         │
│   CSV   │ │  YAML   │ │  JSON   │ │   SQL   │
│         │ │         │ │         │ │         │
└─────────┘ └─────────┘ └─────────┘ └─────────┘

In the above scenario the first of the two API calls, /api/changelog lists the changes sets id and effective dates for the specified projection. The second API call /api/projection/v1/parks requestes the park data at a specific point in time. The requests are routed via a CDN for caching. When the data behind the projections a hook causes the cache to be invalidated. However, it’s completely up to you how the projections are accessed - you could build a RESTful API to expose them over HTTP as above, bundle them in a client side JavaScript module or export them as a set of Apache AVRO files to S3.

Conclusion

Filby transforms how you manage shared, temporal reference data, combining the best practices of source control with runtime flexibility, no matter how the data is consumed. It isn’t a turn key system though, you need to model and provide the data, then write the code calls the Filby API to retrieve the reference data at a point in time and expose it to the outside world.

Ready to explore Filby? Start here.

Guidelines are necessary to strike a balance between autonomy and consistency, ensuring that choices align with organisational goals. But guidelines alone are insufficient. As Jeff Bezos famously said, “Good intentions don’t work. Good mechanisms do.” To address this, I’m excited to introduce eslint-plugin-tech-radar, a robust mechanism to help engineering teams enforce dependency guidelines at scale.

Introducing eslint-plugin-tech-radar?

A traditional Tech Radar provides a visual framework for evaluating tools and technologies based on maturity and strategic alignment. However, it doesn’t prevent engineers from inadvertently or deliberately installing prohibited modules. Mechanisms like private npm registries or post-installation scans have significant drawbacks—blocking transitive dependencies or acting too late in the pipeline.

eslint-plugin-tech-radar solves these problems by integrating directly into your development workflow. It provides:

• Proactive validation: Dependencies are checked against your organisation’s Tech Radar during development, pre-commit, or CI/CD stages.
• Shared configuration: Centralised rules ensure consistent enforcement across repositories.
• Flexibility: Teams can override or adapt rules on a per-repository basis using familiar ESLint escape hatches.
• Version tracking: A built-in “latest” rule ensures your shared configuration stays up to date.

By leveraging eslint-plugin-tech-radar, teams can adhere to established guidelines without sacrificing agility.

Getting Started

Step 1: Define Your Tech Radar

Start by building your Tech Radar. For example:

name,ring,quadrant,isNew,description
prisma,hold,backend,FALSE,Persistence
winston,hold,backend,FALSE,Logging
bunyan,hold,backend,FALSE,Logging
@pgtyped/query,assess,TRUE,Persistence
orchid-orm,trial,backend,FALSE,Persistence
pino,adopt,backend,FALSE,Logging
sequelize,adopt,backend,FALSE,Persistence

Export this CSV as a JSON configuration:

npx --package eslint-plugin-tech-radar -- export-tech-radar \
  --input radar.csv \
  --documentation https://github.com/your-organisation/tech-radar \
  --output radar.json

Step 2: Create a Shared ESLint Configuration

Create a shared ESLint configuration that uses your Tech Radar:

{
  "extends": ["eslint:recommended"],
  "plugins": ["tech-radar"],
  "rules": {
    "tech-radar/adherence": [
      "error",
      {
        "hold": ["prisma", "winston", "bunyan"],
        "assess": ["@pgtyped/query"],
        "trial": ["orchid-orm"],
        "adopt": ["pino", "sequelize"],
        "ignore": [],
        "documentation": "https://github.com/your-organisation/tech-radar"
      }
    ],
    "tech-radar/latest": [
      "error",
      {
        "packages": ["eslint-config-your-organisation"]
      }
    ]
  }
}

Step 3: Enforce Rules Across Repositories

Include the shared configuration in your application’s ESLint setup:

{
  "extends": ["eslint-config-your-organisation"]
}

Run ESLint as part of your CI/CD pipeline or pre-commit hooks to ensure compliance.

Real-World Examples

If your package.json includes a dependency on prisma (a “hold” package), ESLint will flag it:

> eslint .

~/your-application/package.json
  1:1  error  Package 'prisma' is discouraged. See https://github.com/your-organisation/tech-radar for more details  tech-radar/adherence

✖ 1 problem (1 error, 0 warnings)

If your shared configuration is outdated, the “latest” rule will catch it:

> eslint .

~/your-application/package.json
  1:1  error  Package 'eslint-config-your-organisation' must be version 1.0.2.  tech-radar/latest

✖ 1 problem (1 error, 0 warnings)

Additional Tips

Block Installations Using a Preinstall Script

To prevent undesirable dependencies from being installed, run ESLint during npm install:

{
  "scripts": {
    "preinstall": "eslint ."
  }
}

Encourage Healthy Discussion

Changes to the Tech Radar should be accompanied by documented discussions in pull requests or issues. This fosters transparency and ensures alignment.

Conclusion

Autonomy is essential, but so is alignment. eslint-plugin-tech-radar bridges the gap, providing a scalable mechanism for enforcing dependency guidelines. By embedding this tool in your workflow, you can maintain the freedom engineers need to innovate while ensuring their choices align with organisational objectives.

Ready to give it a try? Check out the GitHub repository for more details and start building a better, more consistent codebase today.

Recommended Reading

• Drive

At the time, the School Portal team had a poor reputation. Engineers were reluctant to join, citing a disjointed codebase and a team culture that, while professional, felt siloed and under immense pressure. David’s target wasn’t about deliverables; it was about creating a team environment where people wanted to be. Success would be measured by whether engineers were eager to join us.

Taking Stock

When I first joined the Portal team, I took a close look at the challenges we faced. The problems were easy to spot but significant in scale. The four engineers on the team were working in isolation, each handling months-long initiatives. This approach had been the norm for over a year, leading to knowledge silos. There was no slack. If something broke, only the original developer could fix it, and this would derail whatever feature they were currently working on. The product manager was constantly juggling stakeholder expectations amidst these disruptions, and rapidly losing their confidence.

The quality of the work also left much to be desired. Without collaboration or shared accountability, individual coding styles went unchecked. For example, I know my tendency to overcomplicate solutions needs balancing input from others, but here, such feedback loops were nonexistent. The user experience was no better. Customers frequently reported “bugs” that were actually just confusing designs that violated basic UX principles.

An earlier attempt to fix the codebase had backfired. The team had embarked on a large-scale refactor, but without breaking it into manageable, independent deliverables. They didn’t finish a quarter of the work before their three-month grace period elapsed, leaving the code in an even worse state than before. This was compounded by architectural issues which resulted in frequent timeouts and race conditions.

The planning board was another headache. Hundreds of tickets sat there — bugs, technical improvements, features. It was impossible to prioritise effectively. Meanwhile, bugs were piling up, creating a flood of failure demand (work caused by defects or inefficiencies, as opposed to value demand which directly serves customer needs). One bug in particular highlighted the problem. Over the course of a year, an issue with embedded videos was reported five times by different customers. Each time, it was investigated, deemed low priority, and closed without being fixed. This resulted in a poor customer experience and wasted time not only for our team but also for the company’s helpdesk. Each report required a day of investigation, yet fixing the issue outright would have taken less than two day.

What struck me most, though, was the team’s attitude. They openly disparaged the codebase. It reminded me of a story from “Turn the Ship Around!” by L. David Marquet, where the submarine crew were ashamed of their vessel until they rediscovered pride in their work. The Portal team needed that same shift in mindset.

Turning the Tide

The first challenge was morale. Without hope for change, there would be no progress. Luckily, I had an ace up my sleeve: John Hustler, a colleague and friend. John’s thoughtful approach and willingness to go where he was needed most made him the perfect addition. When he joined, it gave me the support I needed to align the team.

To foster a sense of pride, I introduced a small but symbolic initiative. We created a “Show Portal Some Love” ticket. Every time someone made an improvement, no matter how small, they earned a ❤️. The team may have thought it a bit childish, but my hope was that it gave them permission to make things better, while celebrating small wins and lightening the mood. Slowly but surely, they started to see the Portal as something worth investing in.

The next step was tackling waste. I reviewed every ticket on our bloated planning board, moving low-priority ones to a separate board and closing those that weren’t worth pursuing. The issues I moved weren’t high-priority from the product perspective, but they generated a lot of noise or failure demand. I broke them down into clear, actionable steps and with the help of a borrowed contractor, chipped away at them. Quick wins, like simple UX improvements reduced customer confusion and lowered the volume of reported issues. We also removed unreliable, nice-to-have features, such as a tool for exporting CVs as a single PDF. This feature failed one in ten times, often crashing the system or timing out. Fixing it would have taken weeks, but removing it took minutes, eliminating a major source of frustration for both users and the team.

With fewer interruptions, the team could finally focus. The improved flow created a virtuous cycle: more capacity led to more fixes and a higher standard of work, which led to even greater capacity. Over time, we gained enough breathing room to address deeper issues, like architectural flaws.

Meanwhile, I worked with the product manager and Chief Product Officer to reduce the team’s workload. We dropped one of the large deliverables, freeing up an engineer to focus on improvements. This decision, though difficult, paid off as the team regained momentum.

A New Way of Working

Once the immediate crises were under control, I shifted our approach. Instead of each engineer tackling separate features, we began working together on a single epic. Our first shared effort was integrating social media into the Portal jobs platform. The team was initially skeptical. They worried it would slow them down, but the results spoke for themselves.

Working together meant we could deliver incrementally. Instead of waiting three months to finish three separate features, we completed one feature in the first month, another in the second, and the final one in the third. Not only did this approach deliver value faster, but it also fostered collaboration and shared ownership.

The Results

Within six months, the transformation was evident. The backlog was cleared, the main board had fewer than 25 tickets, and morale was at an all-time high. Six months later, the team had re-architected the application, eliminating performance bottlenecks and race conditions while continuing to deliver product features. We showcased our work at departmental demos, and, for the first time, other engineers were expressing interest in joining the Portal team.

By focusing on collaboration, reducing waste, and building pride, we had turned the team around. The journey wasn’t about individual heroics but about creating an environment where the team could thrive. It’s proof that with the right focus, any team can become a destination team.

Recommended Reading

• Turn the Ship Around!
• The Design of Everyday Things

What’s the Issue?

The new search algorithm prioritises objective sorting criteria like relevance, download counts, dependency counts, and publication date. However, these changes have led to surprising and often unhelpful results in real-world usage. Here are some key concerns:

• Irrelevant Results: Searching for “RabbitMQ” now returns results like vasync and slugid, which are unrelated to RabbitMQ. This behaviour was not observed with the previous search implementation.

• Misranked Relevant Packages: Established and actively maintained libraries have been pushed far down the rankings for relevant keywords, making them difficult to find.

• Prominence of Obscure or Stale Packages: In searches like “hierarchical configuration,” packages that are outdated, rarely downloaded, or both dominate the results, displacing widely used and current libraries.

What might have gone wrong?

From my observations, it appears the new search algorithm may disregard or de-emphasise package metadata such as “keywords” in favour of the contents of the package’s README. The problem with prioritising the README is that it inadvertently boosts irrelevant libraries. For example, many libraries include a section in their README for migrating from previous versions. This causes them to rank highly for searches like “database migration” even when they have nothing to do with databases migration. Similiarly, many libraries include a section titled “Config” or “Configuration” to explain how to set up the package. This means irrelevant libraries frequently rank highly when searching for “configuration”

Demonstrating the problem

To see just how poorly the new search performs for these examples, I searched for “hierarchical configuration” and reviewed the top 10 results…

‘Default’ search order

Only 8 of the above results are configuration libraries, 6 have fewer than 100 downloads and 7 haven’t been published within 3 years. Only config and nconf arguably justify a top 10 spot. More relevant, current and popular libraries like cosmiconfig (61M) and dotenv (42M) do not even feature on the first page.

‘Most Downloaded This Week’ search order

While all of the results are popular and current, only 2 are configuration libraries.

Conclusion

The new npm search algorithm returns either

• mostly relevant, but unpopular and unmaintained libraries, or
• mostly irrelevant, but popular and well maintained libraries

Neither option is useful.

I understand that balancing relevance, popularity, and recency is complex. Furthermore, improving tools like npm search is no small feat, especially given the wide-ranging needs of its user base. I deeply appreciate the work that maintainers to improve npm’s tools. However, the current implementation has clear flaws that undermine its usability.

One of the npm projects authors/maintainers announced the changes via this discussion, and invited feedback. I offered the above analysis three weeks ago, but it was initially misunderstood, and after clarification, has still to be adequately acknowledged. If you’ve encountered similar issues or have ideas for improvement, your feedback could help improve npm’s search experience. Please join the discussion on GitHub.

Thank you.

Many organisations attempt to manage reminders through software calendars like Outlook. However, the user interfaces for such applications are not designed for managing distant events, and if the creator of a reminder leaves the organisation, it may be lost or become read-only.

SAAS platforms that provide automated reminders are another option, but they come with their own challenges. Often, reminders are not a first class features and the automation required to set them up is complex. Furthermore, they may have a per user licensing model or fall within the domain of an IT operations team, making them prohibively expensive or inaccessible. Finally, these services may not be available years into the future, and may lack an export feature.

Ultimately, you cannot rely on third parties to notify you about critical events, on engineers to set reminders, or on organisations to provide systems that make it convenient to do so.

Reminders as Code

Reminders as Code offers an open, convenient and long term solution to this problem. Namely, storing reminders in a text file, committed into to a source code repository. By doing so you gain the following advantages:

• Transparency: The reminders are transparent, versioned, and provided good source control management practices are in place, will be accompanied by a meaningful commit message, so their purpose can be understood after the original committer has left.

• Automation: All good issue tracking systems expose a programmatic API, so with a suitable script, reminders can be created as issues at the appropriate time.

• Notification: Issue tracking systems typically have a wide range of user configurable notification mechanisms, ensuring the reminders do not go unmissed.

• Longevity: Because reminders are stored in an open format, they can easily be migrated if you migrate to a new issue tracking system.

• Scalability: The approach scales across teams and projects without relying on individual ownership or third-party platforms.

• Accessibility: Anyone with access to the source control system and issue tracking system can manage and receive notification of reminders.

Introducing Knuff

Knuff is an open-source reminders as code implementation. Reminders are specified as JSON or YAML and stored in plain text. A script processes the reminders file, identifying those that are due, posting them to the issue tracking system of choice (e.g., GitHub) via a ‘driver’. The script must be run by a scheduler (such as that provided by GitHub Actions) on at least a daily basis.

Example Reminders File

  # The reminder title
  title: 'Update API Key'

  # The reminder body
  body: |
    The API Key expires on the 14th of July 2025.
    - [ ] Regenerate the API Key
    - [ ] Update AWS Secrets Manager
    - [ ] Redeploy the website
    - [ ] Update the reminder for next year

  # Optional description that can be useful to track the background behind the reminder
  description: |
    The API key expires yearly.

  # Schedule a single reminder for 1st July 2025 (See RRULE / RFC5545 format)
  schedule: |
    DTSTART;TZID=Europe/London:20250701T080000
    RRULE:FREQ=DAILY;COUNT=1

  # A set of labels / tags
  labels:
    - 'reminder'

  # A list of repositories to post the reminders to
  repositories:
    - 'acuminous/reminders'

Example Script

import fs from 'node:fs';
import yaml from 'yaml';
import { Octokit } from '@octokit/rest';
import Knuff from '@acuminous/knuff';
import GitHubDriver from '@acuminous/knuff-github-driver';

const GITHUB_TOKEN = process.env.GITHUB_TOKEN;
const PATH_TO_REMINDERS = process.env.PATH_TO_REMINDERS || 'reminders.yaml';

const config = {
  repositories: {
    'acuminous/knuff': {
      owner: 'acuminous',
      name: 'knuff',
      driver: 'github',
    },
  },
};

const octokit = new Octokit({ auth: GITHUB_TOKEN });
const drivers = { github: new GitHubDriver(octokit) };
const knuff = new Knuff(config, drivers).on('error', console.error);
const reminders = yaml.parse(fs.readFileSync(PATH_TO_REMINDERS, 'utf8'));

knuff.process(reminders).then((stats) => {
  console.log(`Successfully processed ${stats.reminders} reminders`);
}).catch((error) => {
  console.error(error);
  process.exit(1);
});

Example GitHub Action

name: Check Reminders

on:
  workflow_dispatch: # Allows manual triggering of the workflow
  schedule:
    - cron: '*/60 * * * *' # Runs every 60 minutes

jobs:
  run-reminder:
    runs-on: ubuntu-latest

    steps:
      - name: Checkout repository
        uses: actions/checkout@v4

      - name: Install dependencies with npm ci
        run: npm ci

      - name: Run Knuff
        run: node index.js reminders.yaml
        env:
          GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}

Limitations

Knuff’s primary limitation is that it can’t force engineers to use it. However, drawing on inspiration from the Kevin Costner movie Field of Dreams: “If you build it, they will come”; Knuff encourages adoption through its simplicity and utility.

A second limitation exists for implementations involving many reminders or repositories. A GitHub Action or Personal Access Token like the one used in the above example may be rate limited, forcing you to register the script as a GitHub App and to implement a more convoluted setup and authorisation flow. See here for an example.

Conclusion

Reliable reminders are not just a convenience; they are a necessity in ensuring the smooth operation of modern software systems. The consequences of missed notifications, ranging from expired keys to overlooked updates, can lead to downtime, financial loss, and unnecessary fire-fighting. Traditional approaches, such as relying on individual engineers or third-party solutions, have proven inadequate due to their inherent limitations, including ill-suited UI, lack of transparency, dependence on specific platforms, and the risk of obsolescence or inaccessibility.

By adopting the Reminders as Code approach, organisations can address these shortcomings. This method integrates reminders into the development process itself, ensuring they are version-controlled, transparent, and long-lasting. Tools like Knuff exemplify how this approach can be operationalised, providing a practical, open-source framework that enables automation, scalability, and accessibility.

The Hidden Danger of Tight Coupling

Operational data stores are not static; they evolve rapidly under active development. Features are added, models are refined, and, in Agile environments, changes are iterative rather than pre-planned. By sourcing data directly from these stores, you tether your reporting and analytics to a system in flux. This creates several critical issues:

1. Constraints on Engineering: Changes to the operational data store must accommodate downstream dependencies. Engineers may become bogged down by these constraints, slowing down the development of new features.

2. Broken Integrations: Without automated tests, changes to the operational store often break models and reports. The resulting chaos is compounded when consumers, often financial and commercial teams, rely on these reports to make decisions.

3. Corporate Fallout: When reports fail, the fallout is rarely trivial. Commercial stakeholders typically wield significant organisational power. Engineering teams find themselves under pressure to implement ad-hoc fixes and facing new layers of process bureaucracy, introducing waste and blocking progress.

A Cautionary Tale: Job Adverts at Tes Global

A clear example of this dynamic comes from my experience with Tes Global, the world’s largest teacher network. At Tes, I led the team developing a jobs board where the lifecycle of a job advert was anything but simple, transitioning between implicit and explicit states of ‘draft’, ‘published’, ‘live’, ‘new’, ‘ending soon’, ‘ended’, ‘closed’ and ‘deleted’. Here’s how they worked:

• A job advert began as a ‘draft’, and was ‘published’ by the advert’s creator.
• Published job adverts automatically becoame ‘live’ after the advert start date.
• Job adverts could be soft ‘deleted’ before going live, but after that, they could only be cancelled since there were associated viewing stats, applications, and potentially charges.
• Job adverts were ‘new’ for the first two days, and ‘ending soon’ for the final two, after which they transitioned to ‘ended’.
• Finally, the job advert was ‘closed’ once the application close date had passed, indicating that no further applications could be submitted.

The complexity was compounded by the absence of an explicitly persisted state for job adverts. This decision was partly because some state change triggers were time-based, making explicit persistence challenging. Instead, the state was derived in application code from various flags and dates. However, we later faced the somewhat unenviable task of reverse engineering the rules and duplicating them in the data lake, with all the reliability issues that entailed.

A Better Approach

A better solution would have been to explicitly model the job advert’s state and export that refined model to the data lake. Real-time events could have fed the data lake while simultaneously unlocking the ability to coordinate and propagate changes across our microservice architecture. Alternatively, data changes (deltas) could have been exported from the database at regular intervals and uploaded to a cloud-based storage service in a text-based, machine-readable format, such as Apache Avro. Both these approaches would also have allowed us to write automated tests to catch breaking schema changes. Although this demands more upfront thought, patience, and effort, it avoids the pitfalls of tight coupling and fragile integrations.

Why Fast and Loose Fails in Big Data

Organisations often opt for the fast-and-loose approach because they believe it offers agility. Every Data team I’ve worked with has request we “just give them everything”, because they are unsure of which reports will be required in the future. This mindset is pervasive but flawed. Poorly modelled data leads to brittle systems, wasted engineering hours, and unreliable analytics.

In contrast, slow and steady wins the race when it comes to data modelling, not least because we are on the brink of a Machine Learning (ML) and Generative AI revolution. However, these technologies remain impotent without high-quality, well-structured data to train on. The upfront investment in thoughtful modelling pays exponential dividends in long-term reliability and usability.

Final Thoughts

The allure of big data has led many organisations to sleepwalk into an architectural nightmare. Directly coupling reporting to operational data stores may seem expedient but ultimately causes more harm than good. By prioritising deliberate schema design, testing, and simple integration, organisations can avoid these pitfalls and unlock the full potential of their data. The choice is simple: sleepwalk into chaos or step deliberately into clarity.