DataOps in the Cloud

While spinning up cloud infrastructure is easier than ever, moving data from an on-prem environment to the public cloud (or from one cloud to another) can be slow, manual, and risky. DataOps can simplify and speed the process migrating data to the cloud.

The Challenge

To realize the agility, flexibility, and scalability benefits of the cloud, enterprises must overcome significant data-related roadblocks. Applications containing sensitive data must be properly secured before being migrated—both to protect against the risk of breach and to ensure compliance with data privacy regulations. Also, cloud migration projects are highly iterative, requiring multiple testing, validation, and rehearsal cycles that depend on the continual availability of fresh, high-quality data.

Enterprises, then, need an effective DataOps approach for moving large volumes of data to cloud environments—one that’s repeatable, secure, and fast enough to keep pace with migration teams’ needs. Any approach also needs to address the full range of migration uses cases demanded by enterprises seeking maximum flexibility: migrating from on-prem environments to a private or public cloud, migration from one cloud to another, or hybrid scenarios involving data synchronized across environments.

DataOps Success Patterns

People

  1. Create a data roadmap: Stakeholders across functions and lines of business must agree on where migrated data will reside as well as how it will be structured, maintained, and consumed.
  2. Set expectations with the business: Migration teams must balance safety and risk reduction with the business’ desire for speed. Communicating concerns to the business and agreeing on a joint action plan eliminates surprises and improves coordination.

Process

  1. Adopt a “data-first” policy: Migrate data in the beginning stages of the overall migration initiative. Easy access to cloud-based datasets early on can preempt application teams bent on pursuing one-off “shadow IT” cloud projects.
  2. Prioritize data quality: When migrating production workloads to the cloud, start with “realistic” datasets. This ensures that every migration phase serves as a rehearsal that builds towards validation of a final production migration.

Technology

  1. Account for sprawl: Seek out technology that not only enables data migration, but also supports follow-on requirements to effectively govern migrated data as it grows, spreads, and otherwise changes over time.
  2. Plan for multi-cloud: Research technologies that satisfy immediate migration plans while also providing flexibility to support multiple cloud vendors in the future.

Data Operators and Consumers

Data friction exists between two groups of people: Data Operators and Data Consumers. Here are some examples of both for Cloud Migration

Data Operators

  • Database Team
  • Storage Team
  • Server Team
  • Information Security

Data Consumers

  • Migration Project Team
  • Developers
  • QA/Testers
  • System Integrators