1.Introduction. What is DataOps

• What is data?
• Structured and unstructured data
• Big Data: 5 V (volume, variety, velocity, verbosity, value)
• World trends of collecting and processing data
• What is Data Analysis, Data Science, Data Governance, Data Quality
• ML and AI in terms of data
• Data + Operations = DataOps

2.Data Storage: RDBMS systems

• Conceptual, logical, physical models
• Codde’s rules to RDBMS
• Basic concepts: Tables, attributes, relationships, keys, constraints understanding
• Normalization (1,2,3 forms; 3.5 and 4 forms )
• ACID: Transactions
• Isolation Levels

3.Data Storage: RDBMS Performance optimization techniques

• What to check? Where to look? What to take into account?
• “Read the query”: execution plans
• Common best practices and quick wins

4.Data Storage: Warehouses

• Normalization Vs Denormalization
• Kimball vs Inmon approaches
• 4 step Kimball approach to build warehouse
• Granularity
• Facts
• Dimensions
• Natural Vs Surrogate keys
• Star and Snowflake schemas
• Data Marts

5.Data Storage: Warehouses. Facts

1. Different types of fact tables:
2. Additive facts, semi-additive, non-additive facts
3. Conformed Facts
4. Factless Facts
5. Transaction Facts
6. Snapshot Facts
7. Cumulative/Consolidated Facts

6.Data Storage: Warehouses. Dimensions

1. Different types of dimensions:
2. Conformed dimension
3. Degenerated dimension
4. Junk dimension
5. SCD

2. Date and Time dimensions

7.Data Storage: NoSQL

• CAP theorem
• BASE transactions
• ACID vs BASE transactions
• Overview of NoSQL main categories

8.Data Storage: NoSQL.

• Key-values databases
• Hadoop and MapReduce

9.Data Storage: NoSQL.

• Column-oriented databases
• Document-oriented

10.Data Storage: NoSQL.

• Full-text search systems
• Graph databases

11.Data Storage: NoSQL

• In-memory databases
• Scaling in NoSQL world: partitioning and sharding

12.Data Storage: Unstructured data

1. How to collect:
2. Blobs
3. Data Lakes
4. Buckets

2. How to process:

• Video
• Audio
• Images

13.Data Collecting: ETL

• Extract, Transform, Load
• Extract, Load, Transform
• Batches vs Row-by-row
• Good and bad patterns
• Data extraction methods
• Data loading methods

14.Data Collecting: ETL. Tools

• SSIS
• Talend
• Informatica
• Pentaho

15.Data Collecting: Streaming

• What is streaming?
• Types of streaming
• Good and bad patterns
• Spark Streaming
• Flink
• Storm
• Kafka Streams

16.Cloud data services. Azure

• Relational data services: SQL Database
• SQL Data Warehouse
• Azure Storage
• Azure Data Lake
• Streaming Services
• NoSQL: CosmosDB

17.Cloud data services. Google

• Relational data services: CloudSQL
• Big Query
• Google Storage
• Streaming Services
• NoSQL: BigTable, Spanner

18.Cloud data services. Amazon

• Relational data services: RDS
• Redshift
• Amazon Storage: S3
• Streaming Services: Kinesis
• NoSQL: DynamoDB
• Comparison: DynamoDB vs CosmosDB vs Spanner
• Comparison: Redshift vs SQL Data Warehouses vs Big Query

19.Data transformations tools

• SQL scripts
• Azure: Databricks
• Google: DataPrep, DataFlow
• Amazon

20.Data processing and preparation

• Basics of ML
• Populate missing values
• Anomaly detections
• Feature building

21.Data Management

• What, why and how?
• What is metadata?
• Data Governance
• Data Standardization
• Data Quality

22.Data Security

• What is sensitive data?
• HIPAA, GDPR and others
• Sync and async encryptions
• Encryption at rest
• Encryption at transit
• Data discovery and classification
• Labeling and Data Visibility

23.Continuous integration and Continuous Delivery

• What is CI?
• What is CD?
• CI with Data-related projects: deploy changes to structures
• CI with Data-related projects: deploy changes to data
• Data and CI: common difficulties