Loading demo links...
What to study for each TBDCDTS domain
Defining Big Data
This section covers what Big Data is, including volume, velocity, and variety, and the distributed architectures used to store and process it. It also covers core Hadoop concepts such as HDFS, replication, and batch versus streaming processing.
Key skills
Big Data characteristics, volume velocity variety, HDFS, NameNode and DataNode, replication, schema-on-read, batch and streaming, horizontal scaling
Configuring a Big Data environment
This section covers configuring Talend Studio for a Big Data environment, including selecting the Hadoop distribution and version. It also covers importing cluster configuration, Kerberos security, NameNode high availability, and using context variables.
Key skills
Hadoop distribution selection, cluster configuration files, Spark configuration, Kerberos authentication, NameNode HA, context variables, connection testing
Managing metadata in a Big Data environment
This section covers creating and reusing Hadoop and Hive metadata in the Talend Repository. It also covers retrieving and synchronizing schemas, validating connections, and propagating metadata changes to jobs.
Key skills
Repository metadata, Hadoop cluster metadata, HDFS and Hive connections, schema retrieval, metadata synchronization, connection validation, change propagation
Managing data on Hadoop and cloud
This section covers reading, writing, listing, and deleting data on HDFS and cloud storage with Talend components. It also covers file formats such as Avro and Parquet, compression, and partitioning data for efficient access.
Key skills
tHDFSInput, tHDFSOutput, tHDFSPut, tHDFSGet, tHDFSList, tHDFSDelete, S3 components, Avro, Parquet, compression, partitioning
Managing data using Hive
This section covers Hive architecture, including the metastore, warehouse data in HDFS, and managed versus external tables. It also covers HiveQL access from Talend, partitioning, bucketing, ORC format, and SerDes.
Key skills
Hive metastore, HiveQL, tHiveConnection, tHiveInput, tHiveOutput, managed and external tables, partitions, bucketing, ORC, SerDe
Manage Big Data Jobs
This section covers designing, transforming, and running Big Data Jobs in Talend Studio, including tMap transformations and subjobs. It also covers tRunJob modular design, error handling, logging and statistics, and safe overwrite practices.
Key skills
Big Data Job design, tMap, subjobs, tRunJob, context variables, error handling, statistics and logs, lookup handling
Managing Spark in a Big Data Environment
This section covers building Talend Spark Batch and Streaming Jobs and configuring them with tSparkConfiguration. It also covers Spark DataFrames, lazy transformations and actions, joins, and caching in Talend Spark Jobs.
Key skills
Spark Batch Jobs, Spark Streaming Jobs, tSparkConfiguration, DataFrames, transformations and actions, tSparkJoin, caching
Managing a Spark cluster
This section covers running Spark on a cluster, including YARN resource management and driver and executor roles. It also covers executor cores and memory, partitioning, and dynamic allocation for efficient cluster use.
Key skills
YARN ResourceManager, cluster and client modes, driver and executors, executor cores, executor memory, partitions, dynamic allocation
Streaming with Talend Big Data
This section covers streaming with Talend Big Data, including Spark Streaming micro-batches and Kafka ingestion with tKafkaInput. It also covers windowing, stateful processing, checkpointing, offset management, and latency.
Key skills
Spark Streaming, micro-batches, Kafka, tKafkaInput, windowing, stateful streaming, checkpointing, offset management, batch interval