116 Data Engineering courses

Register on the Data Engineering with Google BigQuery & Google Cloud today and build the experience, skills and knowledge you need to enhance your professional development and work towards your dream job. Study this course through online learning and take the first steps towards a long-term career. The course consists of a number of easy to digest, in-depth modules, designed to provide you with a detailed, expert level of knowledge. Learn through a mixture of instructional video lessons and online study materials. Receive online tutor support as you study the course, to ensure you are supported every step of the way. Get an e-certificate as proof of your course completion. The Data Engineering with Google BigQuery & Google Cloud is incredibly great value and allows you to study at your own pace. Access the course modules from any internet-enabled device, including computers, tablet, and smartphones. The course is designed to increase your employability and equip you with everything you need to be a success. Enrol on the now and start learning instantly! What You Get With The Data Engineering with Google BigQuery & Google Cloud Receive a e-certificate upon successful completion of the course Get taught by experienced, professional instructors Study at a time and pace that suits your learning style Get instant feedback on assessments 24/7 help and advice via email or live chat Get full tutor support on weekdays (Monday to Friday) Course Design The course is delivered through our online learning platform, accessible through any internet-connected device. There are no formal deadlines or teaching schedules, meaning you are free to study the course at your own pace. You are taught through a combination of Video lessons Online study materials Certification Upon successful completion of the course, you will be able to obtain your course completion e-certificate free of cost. Print copy by post is also available at an additional cost of £9.99 and PDF Certificate at £4.99. Who Is This Course For: The course is ideal for those who already work in this sector or are an aspiring professional. This course is designed to enhance your expertise and boost your CV. Learn key skills and gain a professional qualification to prove your newly-acquired knowledge. Requirements: The online training is open to all students and has no formal entry requirements. To study the Data Engineering with Google BigQuery & Google Cloud, all your need is a passion for learning, a good understanding of English, numeracy, and IT skills. You must also be over the age of 16. Course Content Unit 01: Introduction Module 01: Welcome to This Course 00:01:00 Module 02: How to Get Maximum Value from This Course 00:06:00 Module 03: Course Structure & Coverage 00:02:00 Module 04: Technology in This Course 00:02:00 Unit 02: Introducing Data Warehouse & BigQuery Module 01: Data Warehouse 00:07:00 Module 02: Start With BigQuery 00:05:00 Module 03: BigQuery Web User Interface 00:04:00 Unit 03: First Exploration Module 01: First Data 00:04:00 Module 02: Basic Exploration 00:16:00 Module 03: Functions 00:14:00 Module 04: Common Data Types 00:07:00 Module 05: Different Query 00:01:00 Module 06: Exploring Common Data Types 00:25:00 Module 07: Converting Data Types 00:07:00 Unit 04: Data Flow Basic Module 01: Data Quality 00:06:00 Module 02: Clean & Transform 00:13:00 Module 03: Store Data 00:06:00 Module 04: Upgrading From Sandbox Account 00:01:00 Module 05: Clean & Transform With Dataprep 00:25:00 Module 06: Scheduled Query 00:04:00 Module 07: Analyze Data 00:05:00 Module 08: Data Visualization 00:06:00 Unit 08: Intermediate Query Module 01: Essential BigQuery 00:15:00 Module 02: Load Data into BigQuery (Part 1) - The Basic 00:36:00 Module 03: Tip: Mock Data 00:01:00 Module 04: Load Data into BigQuery (Part 2) - Handling Errors 00:22:00 Module 05: Load Data into BigQuery (Part 3) - Efficient Load 00:14:00 Module 06: Load Data into BigQuery (Part 4) - From Your Data to BigQuery 00:23:00 Module 07: Load Data into BigQuery (Part 5) - In Microservice Architecture 00:20:00 Module 08: Tip: Message Broker Overview 00:08:00 Module 09: Load Data into BigQuery (Part 6) - Recurring Load 00:12:00 Unit 05: Diving into BigQuery Module 01: BigQuery View 00:06:00 Unit 06: Virtual Data using View Module 01: What We Will Learn 00:01:00 Module 02: Google Sheets & BigQuery 00:10:00 Module 03: Google Data Studio 00:13:00 Unit 07: Data Visualization Module 01: Using Join - Theory 00:04:00 Module 02: Using Join - Hands On 00:16:00 Module 03: Union & Intersect 00:06:00 Module 04: Basic Statistical Functions 00:05:00 Frequently Asked Questions Are there any prerequisites for taking the course? There are no specific prerequisites for this course, nor are there any formal entry requirements. All you need is an internet connection, a good understanding of English and a passion for learning for this course. Can I access the course at any time, or is there a set schedule? You have the flexibility to access the course at any time that suits your schedule. Our courses are self-paced, allowing you to study at your own pace and convenience. How long will I have access to the course? For this course, you will have access to the course materials for 1 year only. This means you can review the content as often as you like within the year, even after you've completed the course. However, if you buy Lifetime Access for the course, you will be able to access the course for a lifetime. Is there a certificate of completion provided after completing the course? Yes, upon successfully completing the course, you will receive a certificate of completion. This certificate can be a valuable addition to your professional portfolio and can be shared on your various social networks. Can I switch courses or get a refund if I'm not satisfied with the course? We want you to have a positive learning experience. If you're not satisfied with the course, you can request a course transfer or refund within 14 days of the initial purchase. How do I track my progress in the course? Our platform provides tracking tools and progress indicators for each course. You can monitor your progress, completed lessons, and assessments through your learner dashboard for the course. What if I have technical issues or difficulties with the course? If you encounter technical issues or content-related difficulties with the course, our support team is available to assist you. You can reach out to them for prompt resolution.

Duration 4 Days 24 CPD hours This course is intended for This class is intended for experienced developers who are responsible for managing big data transformations including: Extracting, loading, transforming, cleaning, and validating data. Designing pipelines and architectures for data processing. Creating and maintaining machine learning and statistical models. Querying datasets, visualizing query results and creating reports Overview Design and build data processing systems on Google Cloud Platform. Leverage unstructured data using Spark and ML APIs on Cloud Dataproc. Process batch and streaming data by implementing autoscaling data pipelines on Cloud Dataflow. Derive business insights from extremely large datasets using Google BigQuery. Train, evaluate and predict using machine learning models using TensorFlow and Cloud ML. Enable instant insights from streaming data Get hands-on experience with designing and building data processing systems on Google Cloud. This course uses lectures, demos, and hand-on labs to show you how to design data processing systems, build end-to-end data pipelines, analyze data, and implement machine learning. This course covers structured, unstructured, and streaming data. Introduction to Data Engineering Explore the role of a data engineer. Analyze data engineering challenges. Intro to BigQuery. Data Lakes and Data Warehouses. Demo: Federated Queries with BigQuery. Transactional Databases vs Data Warehouses. Website Demo: Finding PII in your dataset with DLP API. Partner effectively with other data teams. Manage data access and governance. Build production-ready pipelines. Review GCP customer case study. Lab: Analyzing Data with BigQuery. Building a Data Lake Introduction to Data Lakes. Data Storage and ETL options on GCP. Building a Data Lake using Cloud Storage. Optional Demo: Optimizing cost with Google Cloud Storage classes and Cloud Functions. Securing Cloud Storage. Storing All Sorts of Data Types. Video Demo: Running federated queries on Parquet and ORC files in BigQuery. Cloud SQL as a relational Data Lake. Lab: Loading Taxi Data into Cloud SQL. Building a Data Warehouse The modern data warehouse. Intro to BigQuery. Demo: Query TB+ of data in seconds. Getting Started. Loading Data. Video Demo: Querying Cloud SQL from BigQuery. Lab: Loading Data into BigQuery. Exploring Schemas. Demo: Exploring BigQuery Public Datasets with SQL using INFORMATION_SCHEMA. Schema Design. Nested and Repeated Fields. Demo: Nested and repeated fields in BigQuery. Lab: Working with JSON and Array data in BigQuery. Optimizing with Partitioning and Clustering. Demo: Partitioned and Clustered Tables in BigQuery. Preview: Transforming Batch and Streaming Data. Introduction to Building Batch Data Pipelines EL, ELT, ETL. Quality considerations. How to carry out operations in BigQuery. Demo: ELT to improve data quality in BigQuery. Shortcomings. ETL to solve data quality issues. Executing Spark on Cloud Dataproc The Hadoop ecosystem. Running Hadoop on Cloud Dataproc. GCS instead of HDFS. Optimizing Dataproc. Lab: Running Apache Spark jobs on Cloud Dataproc. Serverless Data Processing with Cloud Dataflow Cloud Dataflow. Why customers value Dataflow. Dataflow Pipelines. Lab: A Simple Dataflow Pipeline (Python/Java). Lab: MapReduce in Dataflow (Python/Java). Lab: Side Inputs (Python/Java). Dataflow Templates. Dataflow SQL. Manage Data Pipelines with Cloud Data Fusion and Cloud Composer Building Batch Data Pipelines visually with Cloud Data Fusion. Components. UI Overview. Building a Pipeline. Exploring Data using Wrangler. Lab: Building and executing a pipeline graph in Cloud Data Fusion. Orchestrating work between GCP services with Cloud Composer. Apache Airflow Environment. DAGs and Operators. Workflow Scheduling. Optional Long Demo: Event-triggered Loading of data with Cloud Composer, Cloud Functions, Cloud Storage, and BigQuery. Monitoring and Logging. Lab: An Introduction to Cloud Composer. Introduction to Processing Streaming Data Processing Streaming Data. Serverless Messaging with Cloud Pub/Sub Cloud Pub/Sub. Lab: Publish Streaming Data into Pub/Sub. Cloud Dataflow Streaming Features Cloud Dataflow Streaming Features. Lab: Streaming Data Pipelines. High-Throughput BigQuery and Bigtable Streaming Features BigQuery Streaming Features. Lab: Streaming Analytics and Dashboards. Cloud Bigtable. Lab: Streaming Data Pipelines into Bigtable. Advanced BigQuery Functionality and Performance Analytic Window Functions. Using With Clauses. GIS Functions. Demo: Mapping Fastest Growing Zip Codes with BigQuery GeoViz. Performance Considerations. Lab: Optimizing your BigQuery Queries for Performance. Optional Lab: Creating Date-Partitioned Tables in BigQuery. Introduction to Analytics and AI What is AI?. From Ad-hoc Data Analysis to Data Driven Decisions. Options for ML models on GCP. Prebuilt ML model APIs for Unstructured Data Unstructured Data is Hard. ML APIs for Enriching Data. Lab: Using the Natural Language API to Classify Unstructured Text. Big Data Analytics with Cloud AI Platform Notebooks What's a Notebook. BigQuery Magic and Ties to Pandas. Lab: BigQuery in Jupyter Labs on AI Platform. Production ML Pipelines with Kubeflow Ways to do ML on GCP. Kubeflow. AI Hub. Lab: Running AI models on Kubeflow. Custom Model building with SQL in BigQuery ML BigQuery ML for Quick Model Building. Demo: Train a model with BigQuery ML to predict NYC taxi fares. Supported Models. Lab Option 1: Predict Bike Trip Duration with a Regression Model in BQML. Lab Option 2: Movie Recommendations in BigQuery ML. Custom Model building with Cloud AutoML Why Auto ML? Auto ML Vision. Auto ML NLP. Auto ML Tables.

This course primarily focuses on explaining the concepts of Python and PySpark. It will help you enhance your data analysis skills using structured Spark DataFrames APIs.

This course does not require any prior knowledge of Apache Spark or Hadoop. The author explains Spark architecture and fundamental concepts to help you come up to speed and grasp the content of this course. The course will help you understand Spark programming and apply that knowledge to build data engineering solutions.

Learn the process to design and develop big data engineering projects using Apache Spark. This example-driven advanced-level course will help you understand real-time stream processing using Apache Spark and you can apply that knowledge to build real-time stream processing solutions.

Discover Microsoft Fabric's architecture, master Data Engineering with OneLake and Spark, and elevate your skills in data warehousing and real-time processing. Compare SQL and KQL for better insights, and improve storytelling using Power BI. Finally, you will end with practical data science techniques and data management methods.

Duration 5 Days 30 CPD hours Overview Mining data Manipulating data Visualizing and reporting data Applying basic statistical methods Analyzing complex datasets while adhering to governance and quality standards throughout the entire data life cycle CompTIA Data+ is an early-career data analytics certification for professionals tasked with developing and promoting data-driven business decision-making. CompTIA Data+ gives you the confidence to bring data analysis to life. As the importance for data analytics grows, more job roles are required to set context and better communicate vital business intelligence. Collecting, analyzing, and reporting on data can drive priorities and lead business decision-making. 1 - Identifying Basic Concepts of Data Schemas Identify Relational and Non-Relational Databases Understand the Way We Use Tables, Primary Keys, and Normalization 2 - Understanding Different Data Systems Describe Types of Data Processing and Storage Systems Explain How Data Changes 3 - Understanding Types and Characteristics of Data Understand Types of Data Break Down the Field Data Types 4 - Comparing and Contrasting Different Data Structures, Formats, and Markup Languages Differentiate between Structured Data and Unstructured Data Recognize Different File Formats Understand the Different Code Languages Used for Data 5 - Explaining Data Integration and Collection Methods Understand the Processes of Extracting, Transforming, and Loading Data Explain API/Web Scraping and Other Collection Methods Collect and Use Public and Publicly-Available Data Use and Collect Survey Data 6 - Identifying Common Reasons for Cleansing and Profiling Data Learn to Profile Data Address Redundant, Duplicated, and Unnecessary Data Work with Missing Value Address Invalid Data Convert Data to Meet Specifications 7 - Executing Different Data Manipulation Techniques Manipulate Field Data and Create Variables Transpose and Append Data Query Data 8 - Explaining Common Techniques for Data Manipulation and Optimization Use Functions to Manipulate Data Use Common Techniques for Query Optimization 9 - Applying Descriptive Statistical Methods Use Measures of Central Tendency Use Measures of Dispersion Use Frequency and Percentages 10 - Describing Key Analysis Techniques Get Started with Analysis Recognize Types of Analysis 11 - Understanding the Use of Different Statistical Methods Understand the Importance of Statistical Tests Break Down the Hypothesis Test Understand Tests and Methods to Determine Relationships Between Variables 12 - Using the Appropriate Type of Visualization Use Basic Visuals Build Advanced Visuals Build Maps with Geographical Data Use Visuals to Tell a Story 13 - Expressing Business Requirements in a Report Format Consider Audience Needs When Developing a Report Describe Data Source Considerations For Reporting Describe Considerations for Delivering Reports and Dashboards Develop Reports or Dashboards Understand Ways to Sort and Filter Data 14 - Designing Components for Reports and Dashboards Design Elements for Reports and Dashboards Utilize Standard Elements Creating a Narrative and Other Written Elements Understand Deployment Considerations 15 - Understand Deployment Considerations Understand How Updates and Timing Affect Reporting Differentiate Between Types of Reports 16 - Summarizing the Importance of Data Governance Define Data Governance Understand Access Requirements and Policies Understand Security Requirements Understand Entity Relationship Requirements 17 - Applying Quality Control to Data Describe Characteristics, Rules, and Metrics of Data Quality Identify Reasons to Quality Check Data and Methods of Data Validation 18 - Explaining Master Data Management Concepts Explain the Basics of Master Data Management Describe Master Data Management Processes Additional course details: Nexus Humans CompTIA Data Plus (DA0-001) training program is a workshop that presents an invigorating mix of sessions, lessons, and masterclasses meticulously crafted to propel your learning expedition forward. This immersive bootcamp-style experience boasts interactive lectures, hands-on labs, and collaborative hackathons, all strategically designed to fortify fundamental concepts. Guided by seasoned coaches, each session offers priceless insights and practical skills crucial for honing your expertise. Whether you're stepping into the realm of professional skills or a seasoned professional, this comprehensive course ensures you're equipped with the knowledge and prowess necessary for success. While we feel this is the best course for the CompTIA Data Plus (DA0-001) course and one of our Top 10 we encourage you to read the course outline to make sure it is the right content for you. Additionally, private sessions, closed classes or dedicated events are available both live online and at our training centres in Dublin and London, as well as at your offices anywhere in the UK, Ireland or across EMEA.

Get to grips with real-time stream processing using PySpark as well as Spark structured streaming and apply that knowledge to build stream processing solutions. This course is example-driven and follows a working session-like approach.

Description Data Science Diploma Introducing the Data Science Diploma, an online course tailored for those eager to step into the dynamic world of data science. This comprehensive programme ensures participants grasp the essence of contemporary data science techniques, tools, and theories. At the core of this Data Science Diploma is the module titled Foundations of Data Science. It sets the groundwork by instilling fundamental principles, thereby preparing learners to navigate the expansive sea of data efficiently and effectively. As one progresses, the intricate elements of Data Engineering and Big Data come into play, elucidating how vast amounts of data are managed, stored, and processed. An essential aspect of data science lies in understanding uncertainty and making informed decisions. To this end, Probability and Statistics in Data Science offers learners the tools to decipher patterns, predict trends, and make data-driven decisions. Following closely, Clustering and Classification Techniques provide a deep understanding of how to categorise data into specific groups based on inherent characteristics, paving the way for more precise analysis. But what's data science without the necessary mathematical prowess? The Advanced Mathematical Modeling module hones this skill, enabling learners to craft intricate models that can simulate real-world scenarios. Such models act as the backbone of various data analyses and offer a detailed understanding of the underlying processes. The saying, 'A picture is worth a thousand words,' holds especially true in data science. With the Data Visualisation Principles and Design module, learners are equipped with the knowledge to translate complex data into visually compelling stories. This understanding is further solidified with the Web-Based Data Visualisation Tools, offering hands-on experience in using cutting-edge tools to portray data visually. The course recognises the growing demand for intuitive dashboards that provide real-time insights. The Dashboard Design and Mapping module aids participants in creating interactive and user-friendly dashboards, ensuring stakeholders get a clear and concise view of the data. Yet, as one manoeuvres through these diverse modules, a foundational understanding of computing becomes paramount. Hence, Computing for Data Science takes centre stage, familiarising learners with the computational aspects of data analysis, from algorithms to data structures. Concluding the Data Science Diploma is the module on Domain-Specific Data Science Applications. This segment offers a glimpse into how data science principles are applied across different sectors, from healthcare to finance. It accentuates the versatility of data science, proving its indispensable nature in today's digitised world. To sum up, this online Data Science Diploma ensures a holistic understanding of data science. By intertwining theory with practical application, it equips learners with the skill set required to thrive in the data-driven industries of tomorrow. So, if the realm of data beckons you, this diploma is your gateway to excellence. What you will learn 1:Foundations of Data Science 2:Data Engineering and Big Data 3:Probability and Statistics in Data Science 4:Clustering and Classification Techniques 5:Advanced Mathematical Modeling 6:Data Visualisation Principles and Design 7:Web-Based Data Visualisation Tools 8:Dashboard Design and Mapping 9:Computing for Data Science 10:Domain-Specific Data Science Applications Course Outcomes After completing the course, you will receive a diploma certificate and an academic transcript from Elearn college. Assessment Each unit concludes with a multiple-choice examination. This exercise will help you recall the major aspects covered in the unit and help you ensure that you have not missed anything important in the unit. The results are readily available, which will help you see your mistakes and look at the topic once again. If the result is satisfactory, it is a green light for you to proceed to the next chapter. Accreditation Elearn College is a registered Ed-tech company under the UK Register of Learning( Ref No:10062668). After completing a course, you will be able to download the certificate and the transcript of the course from the website. For the learners who require a hard copy of the certificate and transcript, we will post it for them for an additional charge.

A beginner's level course that will help you learn data engineering techniques for building metadata-driven frameworks with Azure data engineering tools such as Data Factory, Azure SQL, and others. You need not have any prior experience in Azure Data Factory to take up this course.