Unwelcome…
… are platform instability, downtime, {hardware} failure, poor efficiency, cluster useful resource rivalry, repeated course of failures, runaway dwell queries, crucial providers alarms, invisibility into alarm cacophony… the checklist goes on. If these are illnesses you want to treatment …
Welcome!
To this six-part collection, the place we’ll take a look at the right way to get management of the well being of your Cloudera Knowledge platform (CDP) setting. Out of the field, CDP performs fantastically, however over time, if information structure, information engineering, and DevOps finest practices will not be maintained, the Knowledge Metropolis you’ve erected atop a strong CDP bedrock can grow to be the wild, wild, west. Maybe it’s time for some regulation and order to stop additional crimes in opposition to the tech.
Greater than a case examine, we’ve interwoven finest practices gleaned from a number of configurations and consumer websites right into a complete, straightforward to grasp set of directions to diagnose and resolve most of the points that adversely impression CDP environmental well being.
With every weblog we’ll define the signs and root causes of frequent environmental well being challenges and prescribe options. The place we are able to, we’ll embody worthwhile hyperlinks to step-by-step directions to information you thru profitable implementation. After we conclude the collection, we’ll share a homegrown device, an environmental well being scorecard, to observe and handle the well being of your setting.
There are a lot of, many causes that an setting might carry out poorly, and definitely some resolutions take effort and time, however there’s fairly succulent low hanging fruit. Our nice hope is that you simply discover impactful fast wins that encourage you to pursue a number of avenues of well being enchancment. You might also determine to accomplice with our Cloudera Skilled Providers crew who greater than doubled a buyer’s well being rating in two quick quarters.
Classes of CDP Environmental Well being
We’ve categorized features of environmental well being for this collection.
Visibility and Transparency
Into the cluster, platform, providers, and processes. We gained’t be capable of make a lot progress if we don’t have correct visibility into the issues. That’s observability. On this weblog we offer directions and instruments on the right way to acquire visibility, suppress alarm noise, discover and analyze the basis causes of probably the most important alternatives, and proactively notify your customers when incidents happen
Knowledge Asset Standardization
Of frequent datasets, pipelines, processes, and studies. Admittedly, information asset standardization is a multiyear journey; however, addressing solely your most problematic and resource-intensive processes and belongings might yield extra environmental well being enchancment than some other class. We’ll share finest practices on the right way to find and capitalize on these alternatives.
Platform Well being
Consists of {hardware} and providers settings and configurations. Cloudera Knowledge Platform (CDP) have to be configured correctly to perform nicely with excessive efficiency. Moreover, as enterprise wants regularly change, so will your use of the platform, and that can necessitate re-tuning. That can assist you on that journey, we’ll checklist some frequent signs, hyperlink them to root trigger evaluation steps, present correct configuration tips, and description the steps to correctly tune your setting.
The Proper Software for the Job
Consists of the correct use of Impala, CDSW, Airflow, Nifi, and CM. You may be stunned on the adversarial environmental impression of utilizing CDSW as an ETL pipeline device or utilizing Impala to jot down unwieldy queries with an embarrassing variety of joins. We’ve performed it too. We confess. We’ll spotlight the benefits of utilizing Airflow to handle complicated information pipelines with its facility to divide workflow into small impartial duties. We’ll checklist different do’s and don’ts.
Environmental Well being Scoring
Brings all of it collectively by demonstrating the right way to measure, rating, monitor, and management environmental well being by dashboards that we offer for you together with directions to hook them as much as your logs.
Should you’ve bought the signs, the medical doctors are in. Let the therapeutic start!