Those that regularly visit the site know that I focus my professional consulting hours on Data Warehousing, and specifically with Oracle and Oracle Warehouse Builder. However, much of my R&D time I spend researching, downloading, and kicking the tires of Open Source projects I find cool and interesting. Pentaho is clearly one that exists at the intersection between what puts bread on the table and what stimulates the mind, so I accepted their invitation for a week of training in Orlando, FL.

Day 1, like many first days, is mostly introductory, architecture blueprints, team, background, etc. We start getting into some technical details in the subsequent days, but today was mostly an overview. I am continually impressed by this company; not the product per se, because it is a 1.0 product in a very mature market that can only now start to be compared with the “big boys.” However, the company has the mojo to pull this off, I think. They are on a first name basis with CEOs of other key open source companies. Their ranks are filled with former Business Objects and Hyperion recruits. Their board members were senior VPs at Oracle, and on and on. They are building a solid company that is healthy and, from what I can see, can deliver on their vision of an open source BI stack.

What is Pentaho? Pentaho is an open source BI stack that provides the full stack of BI components: Reports, Bursting, OLAP analysis, Dashboards, BAM, etc. Lofty goal indeed… CEO Richard Daley puts it quite simply (paraphrased): We don’t want to be a disruptive technology in just Open Source BI, we want to disrupt the entire BI market place with our technology… It’s a lot of fun… It’s process centric (workflow driven) and has conceded the fact that it won’t be a silo, as the center of the universe. It pragmatically embraces the idea that BI should be part of an overall business process, and that if it is not, then you’re not getting the full value of your BI assets.

This makes profound sense, yes? If your business process of analyzing order fulfillment efficiency is the end of your process (ok, looks like the warehouse in Toronto is 3 times slower than anyone else) then you’re hosed. The process must continue to notify someone of this result, and collaborate on a solution if the intelligence is actionable.

I continue to be critical of their regard for ETL/Data Warehouse as secondary to their platform. I think they have BI covered, and are comprehensive in this regard. What I see, like others, is that the other key piece of “doing BI” is the information integration, cleansing, and transformation. If the data is unintegrated, the business context is difficult to infuse from a “straight SQL query” then you’re in the ETL and Data Warehousing business. That being said, the architecture I’ve seen put forth (haven’t been in the details yet), allows for this relatively easily. Perhaps this will be more robust that it first appears as the details of their product are run through this week.

They released a 1.0 GA in Decemeber. If you haven’t checked it out, DO! It installs in about 2 minutes on Windows (no kidding, it’s one of the best open source demo installations I’ve ever seen).

I was having a discussion with the CEO of an Open Source company recently and we started discussing our opinion of the “kett uhl” Open Source ETL projects. It quickly became clear that we had different ideas about the project and its sponsorship. Until there was a clearly identifying difference… An “ah ha” moment.

Kinetic ETL (open source ETL project) is not the same as KETTLE (open source ETL project).

There’s only a handful of Open Source ETL projects currently. It seems silly that there should already be brand confusion for such a small group of players… Anyone from either of these projects care to comment on their choice of such similar names? Pure coincidence?

Don’t want to get too far into it, because there has been some significant developments from when I last surveyed the scene about 6 months ago. However, I did want to point out that Pentaho has made significant progress in building their Open Source BI stack and are starting to build 1.x release candidates. They’ve sorted out $5 million in financing to really allow them to take this to the next level.

Congratulations on the progress, and I personally look forward to seeing more great work coming from Orlando, FL.

I wasn’t exactly flattering to GreenPlum in this blog. I basically said that anything interesting they were doing was going to be in their MPP (commerical) instead of BizGres (Open Source). I’ve just looked (not used, just read in the docs) at the features for BizGres 0.7 and there are some interesting features in there.

• Bitmap Index Scan Performance Enhancement
• Table Partitioning
• KETL – extract, transform and load (ETL) technology from Kinetic Networks
• JasperReports from JasperSoft

Again, too busy at the mo’ to have a look although would love to see what the KETL is all about. We’ll see what the uptake and community acceptance is… It’ll be interesting to see what happens with the new three way partnership between Jasper/Kinetic/GreenPlum as well.

I’ve used RealVNC for quite sometime, and find that it is a quick and easy method for occasional remote access. Did I mention it’s free?

Since I typically only need it for Windows machines I had only used the Windows VNC Server version. With Linux, “ssh -l user -X myhostname.company.com” would usually suffice. The Windows version polls to check for updates to the window, or screen, or underneath the mouse, etc.

I’ve been duly impressed by the headless VNC server that comes with my White Box Linux. I was expecting a similar experience with delays, pixelation, screen refresh issues but I’ve experienced NONE of that. When I full screen my VNC client I notice little difference than if I were at the console. Anyhow, I just thought I’d mention that the headless Linux VNC is much much better than the Windows polling VNC. Happy network computing!

Every month I review the web traffic reports for my blog, and I’ve always found something rather interesting. Even though I post more information about Oracle and OWB than any other subject, Google seems to send me more traffic from queries like “open source ETL” and “open source OLAP.” You know what they say, customer is king and you gotta give ’em what they want!

In other words, all I needed was just a teentsee weentsee bit of an excuse to take some time to really kick the tires of the open source OLAP server Mondrian.

Some basics… Mondrian is an open source OLAP server, written in Java. It implements an MDX engine, and also exposes an XML/A interface to clients. Mondrian uses a ROLAP architecture, and ends up issuing SQL statements to a JDBC data source to retrieve and calculate. Mondrian works with Access, MySQL, postgres, and Oracle. Refer to the Mondrian architecture pages to get some more information about the architecture.

My overall impressions were positive; it’s a good core set of functionality and performs rather well. Like any Open Source project it is an alphabet soup of supporting libraries, environment variables, generators, frameworks, and takes more than the usual 10 minute commercial product install. We’re not building a kernel here, but it’s not trivial to get the examples up and running.

Mondrian works closely (and appropriately) with an open source implementation of a JSP based Pivot and Charting project, JPivot. The demo for Mondrian includes an example with JPivot querying Mondrian and uses the well known Food Mart demo. I was expecting a bit less from JPivot and was pleasantly surprised that it’s actually rather functional (not commercial product easy to use, but really quite commendable).

JPivot allows for drilling down hierarchies (I don’t think you can use multiple hierarchies) and Pivoting and exhanging the columns and rows. It has a “CUBE” editor that allows you to edit the report. It’s not drag and drop, but definitely works if you “grok” the interface.

Also pleasantly surprising was some pretty decent charting capabilities.

There are some decent selections of charts

In order to provide an OLAP view of your data you have to define some metadata about your Dimensional model (Cubes, Measures, Dimensions) and how they map to your underlying Relational Schema. Check out the samples on the Mondrian site to see how to write your own schema.

Couple of interesting things to point out, Mondrian implements a cache of “relations” used to increase performance. This is interesting because of consistency questions (some fragments are cached, but others are current) but also because it is WICKED fast once it’s loaded into memory. There are some interesting possibilities here, including some work with some distributed P2P OLAP distributed caching research.

I ran across a post by Andrej Koelewijnvia on orablogs.com that made reference to an open source project named CruiseControl.

CruiseControl is a framework for a continuous build process. It includes, but is not limited to, plugins for email notification, Ant, and various source control tools. A web interface is provided to view the details of the current and previous builds.

I’ve worked in environments with automated build processes and think they are absolutely wonderful. There are some significant advantages to an automated build process: