Ford Motor Company is an American multinational automobile manufacturer headquartered in Dearborn, Michigan, United States. In this blog post, we’re highlighting Ford for the DevOps achievements that earned them the ‘Growing DevOps throughout your organization’ award in the2022 DevOps Awards. To learn more about the winners and how they used DORA metrics and practices to grow their businesses, start here.
Ford recognized the potential of cloud computing early on and had approximately 50% of our computing landscape, including CaaS, PaaS, and SaaS, on the cloud. While leveraging the cloud and hybrid environments improved our business operations, none of the platforms we used enforced DevOps practices. Consequently, some teams had console access in production environments and did not consistently adopt basic DevOps and GitOps practices — if they used them at all.
Without proper DevOps practices, our company was exposed to increased risks, such as service issues and the associated costs, delays, and negative customer experiences. As our cloud adoption continued to grow, it became evident that any large-scale migration would introduce even more risks if not effectively managed.
Cloud-first mindset
To maximize the benefits of the cloud, the company realized the importance of adopting a “cloud-first” mindset, where new applications or updated technologies would be developed or migrated to the cloud as the primary option whenever feasible. To achieve this, we recognized the need for a platform that would drive mature DevOps practices.
We set out to:
Enhance customer experience and gain better cost control
Securely deliver power and scalability to teams without resource wastage
Provide business-led app teams with as much direct control as possible
Google Cloud and Terraform at Ford
To solve the above problems, the Ford teams developed a scalable approach to managing project environments through pipelines. These pipelines not only organized the platform but also built and maintained thousands of Google Cloud projects. Together, we and our partners developed a unique solution that restricted console access to sandbox environments only, with system account restrictions that enforced CI/CD pipelines within their extensive community of app team developers.
The pipelines were divided into two components:
The first controlled Google Cloud projects, ensuring that Ford implemented Google-recommended security practices and enterprise technology attestations and policies.
The second pipeline granted business teams sufficient control over their infrastructure, enabling them to dynamically provision and manage infrastructure according to their requirements.
Given that most Google Cloud services support Terraform providers, our Ford Services Teams were able to test and support these providers and create test scripts that all of our DevOps teams could utilize. We believe that Google Cloud’s built-in support for Terraform providers is a game-changer for our company.
To promote the adoption of DevOps best practices throughout the organization, we also relied on our automated provisioning tool: the Ford Cloud Portal (FCP). The FCP enabled teams to rapidly provision resources, facilitated the creation of app-based repositories, provided links to policy attestations and training, and even offered specialized bundles supporting specific personas, thereby accelerating deployments significantly.
The FCP established managed pipelines, where each app team had its own repository to manage infrastructure, fostering true GitOps/DevOps experiences. Through an intuitive interface, the FCP guided app teams through all the steps of the Google Cloud provisioning process, allowing even novice teams to navigate and manage both Google Cloud and infrastructure-as-code environments while reinforcing DevOps practices. Additionally, the FCP provided a list of fully vetted, contractually approved, and Ford-supported Google Cloud services on the Service Selection Screen, enabling teams to support, develop, and share best practices effectively.
The FCP also offered architecture-specific bundles that assisted teams in configuring specific elements based on their use cases. This flexibility allowed teams to choose whether to configure just the project, the project and infrastructure, or even entire development environments. Regardless of the elements configured, they would do so with value-added and validated development and data scientist tooling.
Distributing success to all teams
By mandating the use of pipelines for infrastructure provisioning and management, supported by Google Cloud Terraform services providers, Ford’s app teams embraced the DevOps culture. They no longer had to manage and configure infrastructure through error-prone tickets or interfaces. Instead, they integrated infrastructure provisioning into their pipelines, enabling continuous deployment and testing. This significantly improved our change control and security posture, facilitating faster rollout and rollback of changes. As a result, we witnessed substantial adoption of DevOps practices across our developer community, with over 12% of Ford applications now benefiting from DevOps-managed Google Cloud footprints.
The results have been evident. In 2022 alone, we achieved the following:
Exceeded our application migration targets to Google Cloud with mandated DevOps and GitOps practices
Promoted the use of FCP/Google Cloud pipelines throughout the organization, with approximately half of Ford’s software engineers utilizing pipelines for infrastructure provisioning and management
Moreover, Ford observed notable improvements in dynamic scalability and availability, including in some of DORA’s key metrics:
Deployment frequency: Changes to projects and infrastructure became over 100x times faster, taking minutes instead of weeks, thanks to the migration to Google Cloud.
Lead time for changes: Provisioning velocity in the project pipeline improved tenfold, with changes that previously took over 80 hours on a single channel now being provisioned in less than half an hour.
Change failure rate
Time to restore service:Â
Project capacity: By transitioning from a single channel to parallelized sets, Ford can now handle the enterprise load of project creations or updates per month.
Productivity: Since the shift, Ford has eliminated all manual bottlenecks associated with obtaining Google Cloud projects.
Stay tuned for the rest of the series highlighting the 2022 DevOps Award Winners and read the 2022 State of DevOps report to dive deeper into the DORA research.
Cloud BlogRead More
