Integrate Data Sources to Data Warehouse Appliances Using ELT

Many organizations that we speak with have an interest in leveraging a data warehouse appliance in their infrastructure.  Historically, a data warehouse project was a monumental effort that required very specialized skill, seven figures in capital budget and up to twelve months to get meaningful results from the source.  As we all know, appliances have changed this market dramatically.  Appliances are self -contained units with all of the necessary hardware and software pre-configured.  When the appliance arrives at your data center, you hook it up to your network and you are ready to begin tying in your data sources within 48 hours.

Data warehouse appliances have the ability to handle “big data” volumes-10s or 100s of TBs and even petabytes of information.  The simplicity and speed associated with appliances are other major features our customers value.  However, we at NES maintain that you don’t need multiple terabytes of information to be a candidate for a data warehouse appliance.  You may have a lean IT staff and want a technology that requires minimal maintenance once configured-data warehouse appliances can help.  You may have less than a terabyte of information, but your reports are running slowly and your traditional warehouse lacks the horsepower to meet the business’ SLAs for reporting- data warehouse appliances can help.  You may have the need to conduct analytics that your traditional warehouse has difficulty supporting-data warehouse appliances can help.  So, this is all great and data warehouse appliances sound like they should be implemented in almost every organization to improve efficiency and help them compete.  So where does the messaging become difficult?

The reality of the situation is that data warehouse appliances have come down significantly in price.  We have implemented DW appliances at customers with less than $20m in annual revenue and having less than 1TB of data.  If these organizations can financially swing an implementation like this, most companies should be able to follow suit.  However, the challenge that we see for organizations looking at these types of solutions is managing the data integration layer.  As a small to mid-sized company, for example, investing in a warehouse appliance for the simplicity of management, the cost savings benefits, performance gains and analytics capabilities it can bring to your company is logical.  But, spending six figures on ETL software and additional investment in the specialized services skill-set just to populate your data warehouse appliance can be a tough pill to swallow.  To date, the cost and complexity of integrating data sources has been one of the largest stumbling blocks for companies seeking to implement a warehouse appliance.

NES’ thought long and hard about this problem and came up with a solution.  We created a data integration appliance for IBM Puredata for Analytics (formerly Netezza- and for the purposes of this discussion referred to as “Netezza”) based on the paradigm of extract, load and transform (ELT).  The product is called SQLMaestro  Now you don’t need our appliance to ELT into Netezza for instance.  You can create your own ELT routines.  However, to do this you will need someone with vast knowledge of data integration, a ton of SQL knowledge, expertise with SDLC (Software Development Life Cycle) to manage the code once it’s built and in general someone with lots of time on their hands.  NES has also integrated data quality and US postal name/address verification functions into SQLMaestro.

Our appliance takes all of the SQL logic necessary to bring data sources into Netezza and wraps it with hardware in a “black box” solution that is fully maintained by NES.  The appliance sends transformation commands to the Netezza server and basically leverages Netezza as a massive transformation engine.  SQLMaestro is a great fit for organizations who are just implementing a solution like Netezza, or for companies who are tired of the maintenance and cost associated with ETL technologies.  (There is also the possibility that SQLMaestro can co-exist with ETL solutions depending upon the organizational needs).  SQLMaestro also requires very little training-especially for those who have experience with the graphical designers tied to enterprise ETL tools (Datastage, Informatica, etc.) SQLMaestro’s interface is graphical and resembles these solutions form a UI perspective.    Oh, SQLMaestro is typically 1/3 to 1/2 of the cost of  most ETL solutions, with 10-25X performance improvement.

If SQLMaestro is of interest, the appliance is very easy to evaluate.  NES pre-configures the system to your environment and sends you the physical appliance.  When you receive it, just plug it into your network and away you go.  We also provide a dedicated support person to assist you over the course of the 30 day evaluation.  To view a brief presentation on SQLMaestro, please copy this link into your browser,

NES has been in business as a Premier IBM Business Partner and consulting firm for 34 years.  Our areas of focus are data, data warehousing, data integration and analytics.  To learn more about SQL Maestro/ELT, or how NES might be able to assist you with your information management or analytics needs, please feel free to contact me.

Liam O’Heir

} else {

This entry was posted in Big Data, Business Analytics, IBM Netezza, Uncategorized and tagged , , , , , , , , , , . Bookmark the permalink. Comments are closed, but you can leave a trackback: Trackback URL.
  • Upcoming Events

    There are no upcoming events at this time.
  • About NES

    NES has been in business for 33 years and we’ve always been focused on data. But now, it’s about the liberation of data - with solutions like IBM Netezza, Big Insights and Cognos, we are educating and empowering our customers to make better, more informed decisions about their business and ultimately, to achieve their goals.

Warning: require_once(/home/emconerz/ failed to open stream: Permission denied in /home/emconerz/ on line 501

Fatal error: require_once(): Failed opening required '/home/emconerz/' (include_path='.:/usr/local/lib/php:/usr/local/php5/lib/pear') in /home/emconerz/ on line 501