CCD-410 - Cloudera Certified Developer for Apache Hadoop (CCDH)

Go back to Cloudera

Example Questions

What is a SequenceFile? In the reducer, the MapReduce API provides you with an iterator over Writable values. What does calling the next () method return? Assuming default settings, which best describes the order of data provided to a reducer's reduce method: A combiner reduces: In a MapReduce job, the reducer receives all values associated with same key. Which statement best describes the ordering of these values? For each input key-value pair, mappers can emit: You use the hadoop fs ­put command to write a 300 MB file using and HDFS block size of 64 MB. Just after this command has finished writing 200 MB of this file, what would another user see when trying to access this life? You write MapReduce job to process 100 files in HDFS. Your MapReduce algorithm uses TextInputFormat: the mapper applies a regular expression over input values and emits key-values pairs with the key consisting of the matching text, and the value containing the filename and byte offset. Determine the difference between setting the number of reduces to one and settings the number of reducers to zero. You have just executed a MapReduce job. Where is intermediate data written to after being emitted from the Mapper's map method? When is the earliest point at which the reduce method of a given Reducer can be called? In a large MapReduce job with m mappers and n reducers, how many distinct copy operations will there be in the sort/shuffle phase? Your client application submits a MapReduce job to your Hadoop cluster. Identify the Hadoop daemon on which the Hadoop framework will look for an available slot schedule a MapReduce operation. Table metadata in Hive is: You need to run the same job many times with minor variations. Rather than hardcoding all job configuration options in your drive code, you've decided to have your Driver subclass org.apache.hadoop.conf.Configured and implement the org.apache.hadoop.util.Tool interface. Indentify which invocation correctly with a value of Example to Hadoop? Your cluster's HDFS block size in 64MB. You have directory containing 100 plain text files, each of which is 100MB in size. The InputFormat for your job is TextInputFormat. Determine how many Mappers will run? How are keys and values presented and passed to the reducers during a standard sort and shuffle phase of MapReduce? In a MapReduce job, you want each of your input files processed by a single map task. How do you configure a MapReduce job so that a single map task processes each input file regardless of how many blocks the input file occupies? Analyze each scenario below and indentify which best describes the behavior of the default partitioner? You need to perform statistical analysis in your MapReduce job and would like to call methods in the Apache Commons Math library, which is distributed as a 1.3 megabyte Java archive (JAR) file. Which is the best way to make this library available to your MapReducer job at runtime? When can a reduce class also serve as a combiner without affecting the output of a MapReduce program? What is the disadvantage of using multiple reducers with the default HashPartitioner and distributing your workload across you cluster? You need to create a job that does frequency analysis on input data. You will do this by writing a Mapper that uses TextInputFormat and splits each value (a line of text from an input file) into individual characters. For each one of these characters, you will emit the character as a key and an InputWritable as the value. As this will produce proportionally more intermediate data than input data, which two resources should you expect to be bottlenecks? Which best describes how TextInputFormat processes input files and line breaks? Indentify the utility that allows you to create and run MapReduce jobs with any executable or script as the mapper and/or the reducer? You want to populate an associative array in order to perform a map-side join. You've decided to put this information in a text file, place that file into the DistributedCache and read it in your Mapper before any records are processed. Indentify which method in the Mapper you should use to implement code for reading the file and populating the associative array? A client application creates an HDFS file named foo.txt with a replication factor of 3. Identify which best describes the file access rules in HDFS if the file has a single block that is stored on data nodes A, B and C? You are developing a MapReduce job for sales reporting. The mapper will process input keys representing the year (IntWritable) and input values representing product indentifies (Text). Indentify what determines the data types used by the Mapper for a given job. Determine which best describes when the reduce method is first called in a MapReduce job? Which process describes the lifecycle of a Mapper? You want to understand more about how users browse your public website, such as which pages they visit prior to placing an order. You have a farm of 200 web servers hosting your website. How will you gather this data for your analysis? On a cluster running MapReduce v1 (MRv1), a TaskTracker heartbeats into the JobTracker on your cluster, and alerts the JobTracker it has an open map task slot. What determines how the JobTracker assigns each map task to a TaskTracker? Indentify which best defines a SequenceFile? Can you use MapReduce to perform a relational join on two large tables sharing a key? Assume that the two tables are formatted as comma-separated files in HDFS. To process input key-value pairs, your mapper needs to lead a 512 MB data file in memory. What is the best way to accomplish this? What types of algorithms are difficult to express in MapReduce v1 (MRv1)? For each intermediate key, each reducer task can emit: You have a directory named jobdata in HDFS that contains four files: _first.txt, second.txt, .third.txt and #data.txt. How many files will be processed by the FileInputFormat.setInputPaths () command when it's given a path object representing this directory? What data does a Reducer reduce method process? Workflows expressed in Oozie can contain: All keys used for intermediate output from mappers must: You have written a Mapper which invokes the following five calls to the OutputColletor.collect method: output.collect (new Text ("Apple"), new Text ("Red") ) ; output.collect (new Text ("Banana"), new Text ("Yellow") ) ; output.collect (new Text ("Apple"), new Text ("Yellow") ) ; output.collect (new Text ("Cherry"), new Text ("Red") ) ; output.collect (new Text ("Apple"), new Text ("Green") ) ; How many times will the Reducer's reduce method be invoked? You are developing a combiner that takes as input Text keys, IntWritable values, and emits Text keys, IntWritable values. Which interface should your class implement? Which best describes what the map method accepts and emits? In a MapReduce job with 500 map tasks, how many map task attempts will there be? You want to count the number of occurrences for each unique word in the supplied input data. You've decided to implement this by having your mapper tokenize each word and emit a literal value 1, and then have your reducer increment a counter for each literal 1 it receives. After successful implementing this, it occurs to you that you could optimize this by specifying a combiner. Will you be able to reuse your existing Reduces as your combiner in this case and why or why not? Given a directory of files with the following structure: line number, tab character, string: Example: 1abialkjfjkaoasdfjksdlkjhqweroij 2kadfjhuwqounahagtnbvaswslmnbfgy 3kjfteiomndscxeqalkzhtopedkfsikj You want to send each line as one record to your Mapper. Which InputFormat should you use to complete the line: conf.setInputFormat (____.class) ; ? Which describes how a client reads a file from HDFS? You want to run Hadoop jobs on your development workstation for testing before you submit them to your production cluster. Which mode of operation in Hadoop allows you to most closely simulate a production cluster while using a single machine?

Study Guides