Pentaho 8.2 by Hitachi Vantara is up in sourceforge.net


The second release of Pentaho -I’m still not at ease calling it Vantara- that is made every year, has finally came on December 2nd 2018. It almost seems like a Christmas present to BI enthusiast.
Pedro Alves published (link) a summary that list interesting improvements to Big Data functionality, PDI has a Python Executor and a Jupyter (feed) Integration. CDE can also create push-based streaming dashboards. There are lots of improvements to steps, even BA user capabilities and support of OpenJDK 8 JRE in server in these release.
This are the download links. Installation is done by downloading and unzipping.

Pentaho 8.2 
@ SourceForge
https://sourceforge.net/projects/pentaho/files/Pentaho%208.2/
BA (Web) Server 1.4GBhttps://sourceforge.net/projects/pentaho/files/Pentaho%208.2/server/pentaho-server-ce-8.2.0.0-342.zip/download
Destop tools:PDI (ETL) 1.2GBhttps://sourceforge.net/projects/pentaho/files/Pentaho%208.2/client-tools/pdi-ce-8.2.0.0-342.zip/download
PRD (Reports) 1GBhttps://sourceforge.net/projects/pentaho/files/Pentaho%208.2/client-tools/prd-ce-8.2.0.0-342.zip/download
(optional)PAD (Cube Aggregation Editor)https://sourceforge.net/projects/pentaho/files/Pentaho%208.2/client-tools/pad-ce-8.2.0.0-342.zip/download
PSW (Cube Schema Editor)https://sourceforge.net/projects/pentaho/files/Pentaho%208.2/client-tools/psw-ce-8.2.0.0-342.zip/download
PME (Business Model Editor)https://sourceforge.net/projects/pentaho/files/Pentaho%208.2/client-tools/pme-ce-8.2.0.0-342.zip/download

Good visualizations and good ETL transaformations

Advertisements

Pentaho CDE Samples

This is a collection of Dashboard examples already shared by bloggers that are recreated and tested on Pentaho Server CE 8.1 community edition.</p>

Most of these tutorials, presented in alphabetical order, are still online but some required minor changes to work in the current suite.

You can upload the code to see it in action or use it as reference. At the bottom you’ll find the link and instructions to upload them to your system.

Annual Sales

An interesting Dashboard is “Annual Sales” from Tercer Planeta (post) that shows the table component with columns like the databar or sparkline. It’s also interesting that it uses functions to change extension points, protovis colors or CSS in CCC charts.

The database is AdventureWorks (see previous post to upload it in SQL statements for MySql).

recreación Sales 2003-2005

Bullet Graph Dashboard

The original post of 2012 provided a step by step guide to build this Dashboard that uses a MDX query (Mondrian model from a star schema) and a tabular component to show the monthly data, it also gets that information in Kettle/PDI (ETL) and adds metrics data (like a scorecard/KPI) and shows them on a Bullet Chart. Its a simplified example (processes, bullet graph metrics and aggregations on mondrian) of a real Dashboard that I did for a client. There are also samples in pre-excecution and format functions to modify values in parameters or CSS to change the Chart style.

The chart data is refreshed by updating the queries when the year or month selectors change.

recreación Bullet Graph

For this to work on your system check last section.

Data Dashboard Steels-Wheels

This is a Dashboard that ATOL Conseils & Développements shared a while back. I couldn’t find the tutorial link but they have more complex and beautifull Dashboards in their site or articles like this one that talks about data access in a mondrian cube.

In this example you can see the use of pie and line charts and also the table component. At that time the arrows had to be drawn in javascript (use an if to show a gif file), now its a type of column. There is also code for the tooltip data.

recreación Atol

Sales Overview YTD Dashboard

This one was one of the first Dashboards that I saw. In 2011 Slawomir Chodnicki showed in his blog the complete procedure to build a Dashboard from a draft up to detail javascript that modified a graph (at that time no bullet graph component existed) to show what he wanted. I updated the components but the post is a jewel.

recreación YTD

Steel Wheels Demo Dashboard

This was a Dashboard included in CE. It shows bar graphs updated by a selector.

Demo Steel-Wheels.jpg

Welcome Dashboard in CTools

I guess this is a previus version of the current Dashboard example. The current one even lets you select views so this one could be used as a startup point to lear this tools.

Demo Welcome Dashboard

New Dashboard (RequireJS)

CDE used by default the 24 column blueprint design framework, now its the 12 column, but it also supports Bootstrap, it even has a button to generate a skeleton layout in the Layout view. When you craete a new CDE Dashboard it enables the RequireJS javascript library, you can change to node.js in the settings toolbar. That is important as some events are referended diferently in the former is this.dashboards and in the later is Dashboards with some events and methods available in each one.

This sample is documented at theat Hitachi-Vantara site and is built using the bootstrap framework in layout and RequireJS code. It is built differently that the old ones and further in the tutorial it says it can be embeded in pages outside the server.

Demo Sample with RequireJS

More samples

There are three more examples, two are components to use in more complex Dashboards, one has bars and the other dots.

And the thirth is a report from Will Gorman. This one needs tables from google analytics to work, but its a beautifull sample built with Pentaho Report Designer (PRD).

Will Gorman report.jpg

The Dashboard’s Code

To get this files into your system you have to:

  1. Get this zip file.
    You can confirm that it contains a manifest in xml and a folder with the different CTools files.
  2. Then login into yor Pentaho-server , select the home button and select Browse Files → click on Public folder.
  3. In the right menu the upload option will appear. Click on it. Click on the Browse button and navigate to the downloaded file and click Open and Ok.

After a few momments you can navigate in the Additional Examples folder. To see all the files you have to click on the top menu view and click on Show Hidden Files.

menu

Most of them work with the SampleData Datasource so they should work on your system as it is.

[Edit]

If you want to see Geo Localization examples then check Kleyson de Sousa Rios Github page and download and upload the zip file with his examples for the NewMapComponent (NMC-samples).


* Mondrian schema location (Bullet Graph Dashboard)

I could’t find a way to reference the mondrian schemas on the system JCR repository taht is needed in the Mondrian Input Step, so I edited the Datasource Kettle Component → dsBulletGraph by clicking in (…)

EditInputMondrianCatalog

And used ${Internal.Transformation.Filename.Directory} and move two levels up “/../..” to refer to your tomcat/pentaho solutions.

EditInputMondrianCatalog1

If you develop your PDI transformation, the Catalog location field must contain the complete path-filename to your mondrian XML specification, then afer you upload it in your Pentaho Server, make the edit to your schema path.

So, for this example to work, you have to download a copy of steelwheels.mondrian.xml that’s in your Pentaho Demo system repository to the /tomcat/pentaho-solutions directory, or download this copy in a Zip file.

Pentaho Server 8.1 with MariaDB (MySql)

[Edited on june 13th to add the solution note to Sql parameters]

Migrate your Database

The official way to change the database is at pentaho-wiki and pentaho-doc. These are notes based on my 2011 post that modifies the DEMO to use MariaDB (or MySql) instead of the HSQLDB that comes with it.

Important: I verified this on a Ubuntu 18.04 and MariaDB 10.1, keeping the  jcr (Jackrabbit repository) in thefilesystem and have a problem with JINDI SQL Queries that I have to substitute with JDBC SQL Queries (see the final note).

 

I. DATABASE CONTENT

Driver

[Optional] Update the driver to Connector/J (mysql-connector-java-5.1.46.zip), choose older version and platform independent. Then unzip its content and move mysql-connector-java-5.1.46.jar to ~/Pentaho8/pentaho-server/tomcat/lib. The OPTION error on logs will go away. You should move the older driver to another location.

If you have the desktop tools, copy the driver also to ~/Pentaho8/report-designer/lib, ~/Pentaho8/data-integration/lib, ~/Pentaho8/shema-workbench/lib.

Create content in Database

Start the MariaDB engine and run the suplied scripts in ~/Pentaho8/pentaho-server/data/mysql5, to create required databases:

cd ~/Pentaho8/pentaho-server/data/mysql5
 mysql -u root-user -p root-password < create_quartz_mysql.sql
 mysql -u root-user -p root-password < create_repository_mysql.sql

Download the Sample Database (here), unzip and upload it:

mysql -u root-user -p root-password < SampleData_2018_mysql.sql

II. TOMCAT CONFIG

Quartz Settings

  1. Go to ~/Pentaho8/pentaho-server/pentaho-solutions/system/quartz/
  2. Open quartz.properties.
  3. Find #_replace_jobstore_properties and change it to
    org.quartz.jobStore.driverDelegateClass = org.quartz.impl.jdbcjobstore.StdJDBCDelegate

Hibernte Settings

  1. Goto to ~/Pentaho8/server/pentaho-server/pentaho-solutions/system/hibernate
  2. Open hibernate-settings.xml.
  3. Find the  tag config-file and change it to
    system/hibernate/mysql5.hibernate.cfg.xml
  4. Check settings to MySQL Configuration according to your system (If you use defaults there is nothing to change) in mysql5.hibernate.cfg.xml.

System connections

  1. Go to ~/Pentaho8/pentaho-server/tomcat/webapps/pentaho/META-INF/
  2. Open context.xml
  3. Change classname, url and SQL query to:
    <?xml version="1.0" encoding="UTF-8"?>
    <Context path="/pentaho" docbase="webapps/pentaho/">
    	<Resource name="jdbc/Hibernate" auth="Container" type="javax.sql.DataSource"
    		factory="org.apache.tomcat.jdbc.pool.DataSourceFactory" maxActive="20" minIdle="0" maxIdle="5" initialSize="0"
    		maxWait="10000" username="hibuser" password="password"
    		driverClassName="com.mysql.jdbc.Driver" url="jdbc:mysql://localhost:3306/hibernate"
    		validationQuery="select 1" />
    
    	<Resource name="jdbc/Quartz" auth="Container" type="javax.sql.DataSource"
    		factory="org.apache.tomcat.jdbc.pool.DataSourceFactory" maxActive="20" minIdle="0" maxIdle="5" initialSize="0"
    		maxWait="10000" username="pentaho_user" password="password"
    		driverClassName="com.mysql.jdbc.Driver" url="jdbc:mysql://localhost:3306/quartz"
    		validationQuery="select 1"/>
    
    </Context>

Spring config

  1. Go to ~/Pentaho8/pentaho-server/Pentaho-solutions/system/
  2. Open applicationContext-spring-security-hibernate.properties
  3. Change driver, url and dialect to
    jdbc.driver=com.mysql.jdbc.Driver
    jdbc.url=jdbc:mysql://localhost:3306/hibernate
    jdbc.username=hibuser
    jdbc.password=password
    hibernate.dialect=org.hibernate.dialect.MySQL5Dialect

Add datasources to jndi list

  1. Go to ~/Pentaho8/pentaho-server/pentaho-solutions/system/simple-jndi
  2. Open jdbc.properties
  3. Change the driver definitions to mysql
    SampleData/type=javax.sql.DataSource
    SampleData/driver=com.mysql.jdbc.Driver
    SampleData/url=jdbc:mysql://localhost:3306/sampledata
    SampleData/user=pentaho_user
    SampleData/password=password
    Hibernate/type=javax.sql.DataSource
    Hibernate/driver=com.mysql.jdbc.Driver
    Hibernate/url=jdbc:mysql://localhost:3306/hibernate
    Hibernate/user=hibuser
    Hibernate/password=password
    Quartz/type=javax.sql.DataSource
    Quartz/driver=com.mysql.jdbc.Driver
    Quartz/url=jdbc:mysql://localhost:3306/quartz
    Quartz/user=pentaho_user
    Quartz/password=password
    SampleDataAdmin/type=javax.sql.DataSource
    SampleDataAdmin/driver=com.mysql.jdbc.Driver
    SampleDataAdmin/url=jdbc:mysql://localhost:3306/sampledata
    SampleDataAdmin/user=pentaho_admin
    SampleDataAdmin/password=password

Stop HSQLDB from starting

  1. Go to ~/Pentaho8/pentaho-server/tomcat/webapps/pentaho/WEB-INF/
  2. Open web.xml
  3. Find the word HSQLDB. There are to points to edit. You will delete “–>” on the first line and “<–” on the last, to make it into a comment.
    In version 8.1 go to lines 25 and 167. They should look like this:

    <!-- [BEGIN HSQLDB DATABASES]
     <context-param>
     <param-name>hsqldb-databases</param-name>
     <param-value>sampledata@../../data/hsqldb/sampledata,hibernate@../../data/hsqldb/hibernate,quartz@../../data/hsqldb/quartz</param-value>
     </context-param>
     [END HSQLDB DATABASES] -->

    and

    <!-- [BEGIN HSQLDB STARTER]
     	<listener>
     	<listener-class>org.pentaho.platform.web.http.context.HsqldbStartupListener</listener-class>
     </listener>
     [END HSQLDB STARTER] -->

III. Change Web App Datasources

  1. Start your server and log in your browser (localhost:8080/pentaho) as Admin
  2. In the PUC (Pentaho User Console) page click at the left top menu: FileManage Data Sources
  3. You need to select SampleDataJDBC data-source and click on the gear icon to open the config window.
  4. Change the database type to MySql. Then change the Database Name to sampledata (the database name is in lowercase), port to 3306, type again the password (password) and test the connection. Click OK if its succesfull.
    ManageDatasource_v8.1

Here, you can define a connection to your databases or you can return later.

That’s it, you should have a working Server with MariaDB. To test it by using the samples: Browse: Samples → Public → Steel Wheels.

 

IV. Modify your SQL Queries that uses Parameters

But there is a problem if you want to use a query wit SQL with parameters. Maybe there is a missconfiguration in CDA or the repository, or a problem with the MySql driver for a MariaDB engine, as the parameters in the JNDI configuration report an error and do not work.

To get a result I use only SQL JDBC, by supplying these config parameters on each query

driver:   com.mysql.jdbc.Driver
 url=      jdbc:mysql://localhost:3306/sampledata?generateSimpleParameterMetadata=true
 user:     pentaho_admin
 password: password

 

Troubleshoot

If you have problems check the log file is ~/Pentaho8/pentaho-server/tomcat/logs/catalina.out its very verbose so look for ERROR lines

Pentaho 8.x

[Edited on may 15th 2018 for 8.1]

In sourceforge, Pentaho has been renamed “Hitachi Vantara | Pentaho”, the binaries for all projects in this version has been uploaded for the community. Pedro Alves has summarized the main features for 8.0 and 8.1.

The links to the Web server and the Desktop tools are:

Pentaho 8.1: https://sourceforge.net/projects/pentaho/files/Pentaho%208.1/
Pentaho 8.0: https://sourceforge.net/projects/pentaho/files/Pentaho%208.0/

  1. Server
    • Tomcat server to squedule jobs, grant access to run reports and jobs to users and design dashboards.
      BI Server 8.1
      BI Server 8.0 [1.2GB]
  2. Client Tools
    • Pentaho Data Integration – Best ETL you’ll find.
      PDI 8.1 [1 GB]
      PDI 8.0 [979.8 MB]
    • Pentaho Report Designer. Reporter for different databases.
      PRD 8.1 [868.7 MB]
      PRD 8.0 [666.0 MB]
    • Pentaho Metadata Editor – Grant access and model (physical and business) on how to query your data.
      PME 8.1 [861 MB]
      PME 8.0 [836.5 MB]
    • Pentaho Aggregation Designer – Specify mondrian cube aggregations.
      PAD 8.1 [26.3 MB]
      PAD 8.0 [25.6 MB]
    • Pentaho Schema Workbench – Edit your mondrian cube.
      PSW 8.1 [31.3 MB]
      PSW 8.0
      [84.4 MB]
    • Big Data Plugin
      BDP 8.1 [629.5 MB]
      BDP 8.0 [619.8 MB]

Sound like its time for discovery.

Hitachi Vantara

Today September 19, 2017, Hitachi Vantara was announced, it is a new business entity that will unify the operations of Hitachi Data Systems, Hitachi Insight Group, and Pentaho.

So we’ll be saying both Pentaho and Vantara for a while to refer to this BI suite.  🙂

This is the press release: https://www.hitachivantara.com/en-us/news-resources/press-releases/2017/gl170919.html

The main new Site: https://www.hitachivantara.com/

And new twitter account to follow: https://twitter.com/HitachiVantara

The Community Edition at Sourceforge: https://sourceforge.net/projects/pentaho/ with the Hitachi Logo and title “Hitachi Vantara | Pentaho”. The wiki in http://wiki.pentaho.com/display/COM/Community+Wiki+Home

Data integration is still “PDI” or “Kettle” in the community area: https://community.hds.com/docs/DOC-1009855

It seems (#HitachiNEXT) they’ve got a nice business and consulting strategy. Lets see what they do with the original Open Source philosophy.

Pentaho 7.1 – Demo Install

The newest release Pentaho 7.1.0.0.12 in its community edition is available for download at sourceforge (link) since the 22th of may of 2017.

Pedro Alvez announced its availability the same day with an excellent post in his blog (link). He mentions new visualizations, scalability with big data engines and repository improvements -someday they’ll deliver something you can work confortable with, maybe-, and a new web theme. We’ll see how they are supported in the CE edition. A new mobile access for the EE. The PDI team is exited about its ‘metadata injection’ improvements.

You’ll find it at Sourceforge.net:

  • Web application: Business Intelligence Server [1.1 GB] with 58 closed reports.
  • The best ETL aplication: PDI [904 MB] with 142 closed reports.
  • Report Designer [606 MB] with 19 closed reports in their JIRA system.
  • Pentaho Metadata [787 MB] with 5 closed reports.
  • And Big Data Shims folder.

Steps for backup, install, upgrade to a MySQL local DB are the same that you are already familiar with and has been previously posted in this blog (link) and well documented at the Pentaho Wiki.