While we are working on Eclipse SCADA 0.2.0, we had to make some breaking changes on the openSCADA build system.
After most of the code has been migrated from openSCADA to Eclipse SCADA, the massive build system that was present became quite an issue bringing out new builds of openSCADA itself. openSCADA is an extension to Eclipse SCADA and brings the OPC and SNMP driver, together with packages for Postgres and the necessary configurator plugins for the IDE.
So today we pushed the changes which will break the old build system. This means that openSCADA 1.2.0 will no longer be built and published to “download.openscada.org”, “repo.openscada.org” or the other “apt” and “yum” domains. Instead of having a multi part build which requires a lot of manual interactions when creating a release build, we switched to using Maven Tycho and an aggregated build, bringing out all parts of openSCADA at the same time.
In order to preserve the existing download artifacts, the old domains will be available, but won’t be updated any longer. Also will the nightly builds of 1.2.0 be kept available. But they won’t be updated anymore. And also the upcoming 1.2.0 release will be done using the new build system.
What would have been openSCADA 1.2.0 is now Eclipse SCADA 0.1.0.
Have a look at the release announcement at eclipse.org.
We intend to make an openSCADA release 1.2.0 in the next few weeks. It will contain the additions to Eclipse SCADA 0.1.0:
- OPC Driver
- SNMP Driver
- REST Implemenation
While openSCADA and Eclipse SCADA already have some simple value archive that can store values for years, including master/slave replication and composite servers, we know that the archive won’t scale beyond the boundaries of one server.
That is why we started thinking of having one “big data” like historian for Eclipse SCADA. We started playing around a little bit with HBase and came up with some pretty nice ideas. Hadoop and HBase as a backend allow us to scale the way we want to.
Since we started by building an extension to our current value archive system, we started the development inside the Eclipse SCADA project, and are still tracking some design thoughts there. However it became obvious very soon that this system would not only work for Eclipse SCADA, but could also be used by other systems as long as we keep the interfaces as open as possible. So hopefully this project will end up as a new Eclipse project beside Eclipse SCADA.
So what is the current state of all this? It is an early development stage! We do have a specific use case in the moment that we want to build and for this it seems to be ready. We also have a lot of ideas in our minds that could be implemented. For the moment our focus is on realizing our use case and once we achieved that, we want to make a first release of the source code.
Here is what already works:
- Creating value stores and storing data in raw format
- Compacting raw data to more efficient storage formats
- Extracting data from the storage using a CSV format
- Query before and after the query region
Things that we need to do first:
- We do want to compress doubles depending in their value. Since this will influence the storage format, we do want to make this as soon as possible.
- Create a build system. Building from the IDE is ok for some time. But not for long.
- Create a HTTP based collector framework with buffering
- Implement an Eclipse SCADA collector module
- REST API
- Extend the Native API
- Lots more…
Haystack data exported to Eclipse Birt using the HTTP CSV servlet.
A simple Eclipse RAP based Query UI
The output of the CSV export servlet
Finally we do have some MSI packages for the windows platform again. In the past we used AdvancedInstaller as our installation tool for the OSTC (openSCADA Admin Client). But this tool is not open source and therefore does not fit into the Eclipse SCADA project. As an alternative we use the Wix Toolset now.
The process of creating MSI files using Wix needs to be performed on some Windows machine, which is the reason why this is no automated process right now. But we will create MSI files for the milestone releases. On the bright side these MSI files will now be signed by the Eclipse Foundation, after bug 426371 has been resolved.
It was there is in the past, a long time ago. A console view that traces changes of a data item, in detail. Now it is back, but this time based on the Eclipse Console framework which solves all the issues the previous version had. It is available in the recent version of the Eclipse SCADA Admin Client.
This is something we wanted to do a long time ago. But now we needed it … a wireshark dissector for the Eclipse SCADA NGP protocol stack. You can get the source from github (ctron/eclipse-scada-ngp-dissector).
Due to the complexity of the NGP protocol stack, we were only able to implement to message channel layer of the protocol. Also the streaming compression and SSL/TLS are not supported at the moment. But on the bright side it can handle TCP fragmentation and multiple packets in one TCP packet. Hopefully we can extend it a little bit in the future. For the moment it is a good start.
On friday we released M2 of the ongoing development towards 0.1.0. It contains new features and bug fixes. We also tested the whole new release workflow for the second time.
The most interesting new features might be the extensions to modbus and the REST API.
It was hard work. But we finally have started the release cycle for 0.1.0 with the first milestone build today.
The documentation of how to perform the release build is here in the Eclipse Wiki.
The “adminclient” (formerly “ostc”) can be downloaded here:
The yum repository is here: http://download.eclipse.org/eclipsescada/repos/milestone/0.1.0/yum/
The debian packages (currently no APT repository): http://download.eclipse.org/eclipsescada/repos/milestone/0.1.0/deb/
The P2 repository is here: http://download.eclipse.org/eclipsescada/updates/milestone/0.1.0/
After the code base has been committed and nearly everything is in the integration builds we started pushing the artifacts to download.eclipse.org/eclipsescada. The composite P2 repository for integration builds is now at http://download.eclipse.org/eclipsescada/updates/integration/. This is a P2 repository location!
But beware! This is not a release, these are only the most recent integration builds from the build server.
The majority of the code base of openSCADA is now contributed to the Eclipse Foundation and will by checked by the IP team in the next weeks. This process will take some time since we have a rather larger code base. Fortunately we are in the incubating phase and can take advantage of the parallel IP process, so after an initial check we are already allowed to check in the code. So this means that in a few days the majority of the code base will be in the Eclipse SCADA GIT repositories and will be built on your hudson instance.
The major tasks now will be to restore all the nice deployment options we have with openSCADA, like the MSI installer for OSTC and the configuration system that will generate “ready-to-run” debian and rpm packages. We hope that we can offer an apt and yum repository at Eclipse like we have it now for openSCADA.
For the moment the new version of OSTC (called ESAC) can already be downloaded from the continous build server. However this is not a release, but a work in progress.
So it looks like we can stick to our plan and target a first release of Eclipse SCADA 0.1.0 for the end of the year.