Sunday, October 10, 2010

Incorporating scala, java, sbt, JOGL, Qt, and Ruby/Python

For several years now, I've been iterating on a small project that allows the user to build a Sunflow scene file. When starting it, I tried to effectively design the program with the write technologies for the task.

The language of choice
First, Sunflow is written in java and as such runs on the JVM. Not only do I want to be able to build a Sunflow file, but I want to render it interactively using Sunflow's libraries. There are many langauges available on the JVM right now, but only a few sparked my interest. First, I had done everything in Java previously, which was okay but was at times a little tedious. The languages I looked at as possible replacements were scala and clojure. Although I was very interested in clojure, I thought at the time of starting that it was too foreign to me and I would end up doing something extremely naive in my design.

Scala is a statically-typed language that can at times closely matches the java language. Scala has built a large type-system, which includes many data structures that are both mutable or immutable depending on what you're using them for (functional programming, for example, would probably favor the immutable structures). Scala treats it's functions as first-class citizens, so they're easily passed around to other functions. I can even make large anonymous functions that fit right within another function call. It's syntax is also slightly less verbose than java, and it provides an excellent getter/setter infrastructure that's extremely clean.

Scala isn't all roses, however. First, scala seems to be growing faster than my app, so as I tried to stay up to date with the latest scala features, it's often broken compatibility in significant ways, which has been a bit of a distraction fixing. Second, some of the code is illegible to me. It has a complicated type system and weird language symbols sometimes that seem to pull from esoteric languages for the sake of it instead of creating a simpler style. My scala is probably pretty java-like, so it's fairly easy to read.

Building with scala
Another advantages of scala is its ability to cross-compile with java. I was previously using ant, which was my foray into a java build system. First, I don't like work in XML for a myriad of reasons. After the ant scala compiler tool started throwing bogus errors, I was advised to move over to the simple-build-tool or sbt.

After adding a launcher to my class path, sbt allowed me to quickly setup a new project, whose project file is written in scala, a much more preferred way to configure my app. I simply put all my scala code in ./src/scala and all my java code in ./src/java and sbt combines them. Although it took a moderate amount of time to setup my project, in the end it's been a joy to work with sbt.

JOGL
The JVM requires JOGL to do OpenGL rendering. This can be tedious to setup as I'm used to just tossing a jar onto the classpath and having my build system include it. Because JOGL requires native libraries, which need to be included separately to the java library path. This took a while for me to figure out, but ended up being a simple setup in the end.

import sbt._
import java.io.File

class SunshineProject(info:ProjectInfo) extends DefaultProject(info)
{
  // tells sbt which class to run
  override def mainClass:Option[String] = Some("com.googlecode.sunshine.Sunshine")

  def nativeJOGL:String = {
    var os = System.getProperty("os.name").toLowerCase
    var arch = System.getProperty("os.arch").toLowerCase

    // this is to match the JOGL builds
    if (arch.matches("i386")) arch = "i586"

    if (os.contains("windows")) {
      os = "windows"
      arch = "i586"
    }
    println("OS: %s".format(os))
    println("JOGL Path: %s".format("./lib/jogl-2.0-%s-%s".format(os, arch)))

    "./lib/jogl-2.0-%s-%s".format(os, arch)
  }

  override def fork = forkRun("-Djava.library.path=%s".format(nativeJOGL) :: Nil)


All this basically does is query the OS and architecture from the JVM and adds it's respective JOGL directory to the library path. This required me to fork the JVM, as adding paths to the current JVM doesn't seem to work. Notice I just had to override the fork command and sbt knows I want to fork. Overriding the run command won't fork the JVM when starting. I did something similar in ant, but it was much longer and more difficult to read.

Using Qt
Java Swing is usually an okay GUI library for my small java projects, but it's slightly cumbersome when trying to do something complex. Trolltech's Qt is a very popular framework that has continually gained popularity over the years for it's great documentation, intuitive API, and it's event handling system.

QtJambi is the java binding for using the Qt libraries. A few months ago Trolltech dropped support of QtJambi, but pushed it off to the community to continue updating. So far, they seem to be doing a decent job, although they have been continually asking for help.

sbt supports automatic library manage via Apache Ivy. Instead of shipping every build of Qt for a given architecture, I can setup Qt as a managed library. By pointing sbt to the QtJambi servers, sbt will automatically fetch them during compilation.

val qtDepSnapshots = "Qt Maven2 Snapshots Repository" at "http://qtjambi.sourceforge.net/maven2/"
val qtDep = "net.sf.qtjambi" % "qtjambi" % "4.5.2_01"
val qtjambiBase = "net.sf.qtjambi" % "qtjambi-base-linux32" % "4.5.2_01"
val qtjambiPlatform = "net.sf.qtjambi" % "qtjambi-platform-linux32" % "4.5.2_01"

Right now, I'm only fetching Linux x86 libraries as that's what I'm working off of. Adding the block above directly into my build class will tell sbt to grab them for me. There's a bit of magic going on here for me as I don't understand how sbt knows these variables are library dependencies or just values I've created in my program. Regardless, it's enough to get Qt downloaded and onto the classpath.

Incorporating a scripting language
At this point, I could just start coding my application, but since I'm doing a lot of designing on the fly, it takes a long time to compile my application to see a small change. I wanted to incorporate a scripting language that lets me make changes to the interface quickly without existing the program.

As much as I complain about python, I use it a lot at work and am fairly productive programming in it. The jython project provides a python implementation that runs on the JVM. I've used this library before a year ago, but had to scrap it as it was too slow latency-wise. I've heard it's gotten significantly faster recently, so I gave it another shot and found it to be much faster. Working with Qt, however, seemed to turn up a bug blocker.

[error] Exception caught after invoking slot
[error] Traceback (most recent call last):
[error]   File "", line 15, in 
[error]   File "", line 9, in __init__
[error] TypeError: Proxy instance reused


I was initially wondering if working with Qt from a scripting language was not stable, however, this seemed to be only a jython issue. I reported it to the jython team, which seems to be resolving the issue right now for their upcoming build.

In the mean time, I thought I would give ruby a spin. I'm rather unfamiliar with the Ruby language, although I haven't been oblivious to the huge success it has garnered in web community with Ruby on Rails. I've also heard great things about JRuby, the Ruby implementation on the JVM. At one point it was actually faster than the C++ build of Ruby, although I'm not sure this is necessarily true anymore.

The ruby language seems to be something between python and perl, although that's another comment that will possibly get me shot by a Ruby developer. It's a purely object-oriented language--more so than python--and provides a bit more syntax flexibility than python (for better or for worse) including some parsing syntax from perl.

val factory = new ScriptEngineManager()
val ruby = factory.getEngineByName("jruby")

val urlFile = "clear_scene.rb"
val url = getClass().getResource(urlFile)

ruby.eval(new InputStreamReader(url.openStream()))

With my main scala code, I open a Qt main window. Then hand things over to jruby. My ruby code clears out the window and fills the UI programmatically. I can add menus and event handlers as I go and when I'm ready to see the change, I simply restart the ruby evaluator and my UI rebuilt instantly with no recompilation. I literally see no latency continually reloading my ruby code and seeing my interface changes change on the fly. I plan to do most of my designing in ruby and gradually move the classes I create over to the scala-side once they become stable for a performance boost.

I like to keep one terminal open running sbt with the "~copy-resources" action continually copying my ruby changes over to the build path, while the other terminal compiles and runs my app as I go through the code changes in scala/ruby.  

I've just barely started with it, but I've been enjoying ruby for the most part. A ruby developer would probably say I'm programming like a python programmer (a scala developer would probably say my scala looks like java). Anyway, I haven't had any road blocks importing the various Qt or scala classes besides two minor inconviences.

scala hashmaps
The classes I've built in scala are compiled to byte code and I've been able to read them from java and jruby without any problem. However, I use a few hashmaps in scala and I'd like to be able to iterate through them in my ruby code. JRuby provides some hooks to iterate through java collections, but not for scala, so I made a simple wrapper for my scala hashmaps so I can iterate through them in ruby normally.

class ScalaHashMap
  def initialize(hash)
    @hash = hash
  end

  def each
    @hash.size().times{|key| yield key,@hash.get(key).get}
  end
end

Getting QtJambi to see the JOGL Context
JOGL provides some useful widgets that directly tie into java swing. These widgets don't exist in JOGL for Qt. As such, it was slightly confusing trying to find documentation on getting a GL3 context from JOGL while inside a QGLWidget.

class PanelGL < QGLWidget   
  def initialize(parent = nil)     
    super     
    @camera = Register.cameras.get(0).get      
    profile = GLProfile.get(GLProfile::GL3)     
    glCaps = GLCapabilities.new(profile)     
    glCaps.setPBuffer true     
    @pBuffer = GLDrawableFactory.getFactory(profile).createGLPbuffer(glCaps,DefaultGLCapabilitiesChooser.new(),       1, 1, nil)   
  end   
  attr_reader :camera    

  def initializeGL     
    @ctx = @pBuffer.getContext()   
  end   
  attr_reader :gl    

  def resizeGL(width,height)     
    @ctx.makeCurrent() # this line isn't required for the JOGL Swing components     
    gl = @pBuffer.getContext().getGL.getGL3     
    gl.glViewport(0,0,width,height)   
  end    

  def paintGL     
    @gl = @pBuffer.getContext().getGL.getGL3     
    @gl.glClearColor(0.3, 0.3, 0.3, 0.0)     
    @gl.glClear(GL.GL_COLOR_BUFFER_BIT | GL.GL_DEPTH_BUFFER_BIT)                  
    @gl.glEnable(GL.GL_DEPTH_TEST)      

    ...      

    @gl.glDisable(GL.GL_DEPTH_TEST)   
  end 
end 

I first create an pBuffer, so I can use it's context across GL widgets and share things like display lists and VBOs. The big difference in the Qt code is having to call makeCurrent() in the first GL callback that my program executes, which happens to be resizeGL. Calling makeCurrent() there beforehand makes sure the GL context is running on the same thread as the Qt GUI (or vice versa, I guess).

Conclusion
That's my basic setup and now I've got a lot of coding to do.

16 comments:

Zab Rab Oof said...

I'm currently coding a scala-jruby-opengl project, and I was about to consider adding Qt to the lot.
Thank you! Finding this blog made me feel less alone :)
Initially I used only scala, but jruby is really good and reloading the ruby sources when they change really makes development a lot more pleasurable, in comparison with scala's very long compilation times. Right now the majority of my project (a game) is in ruby, with scala being used only for time-critical parts.
Good luck for your project!

strattonbrazil said...

Thanks! Let me know how you end up working with Qt connections in JRuby.

Unknown said...

Have you had to make any more changes to keep up with library updates since 2010? can you perhaps post a minimal jogl hello world in scala on github or the like? i'm banging my head against the wall trying to learn scala, jogl, sbt, and the weird hacks to get it all going...

Unknown said...

Nice post, thanks for providing best services about the clipping path. We like to implement more and more services.

clippinpath099 said...

Through images send the message all over the world.It support all the business related work. So, make your images perfect by using clipping path supported.

clippinpath099 said...

For making web application different languages are used. For supporting all these languages pen tool make the image using clipping path techniques. According to the requirement of applications experts make and send the message to the viewers. These pen tool used by clipping path unique and make the perfect images.

Shirley(clipping path) said...

Oh my!!! You are just too true to be good. How do you manage to write and research on such wonderful things? You have inspired me to work harder now. I shall try as much as possible to enjoy life to the fullest and be satiated with the wonderful things that are around me, which I have been unaware of until now.

clipping path bangladesh

Anonymous said...

Thanks for the post and sharing the blog. Valuable and excellent post, as share good stuff with good ideas and concepts.
lots of great information and inspiration. I just would like to say thanks for your great efforts.
I appreciate your excellent post.Clipping path service India

Unknown said...

We have read the blog and it's a nice thing in your blog. Really helpful with the information provided.Thank you for the post. For more information. Visit: Picture Manipulation

http://www.designercountry.com/ said...


I really enjoyed reading your blog, you have worked very well on this website but also I like it and I have received a lot of suggestions in your web site and I have learned all about it and I have been benefited from writing such blogs. Stay

Unknown said...

FixiPixi provide best clipping path service.

Edgar C. Jones said...

Thank you so much for sharing your informative article. I appreciate it.
Photo restoration service || Best Clipping Path service || Image masking
Best Ghost Mannequin Services || Car Image editing || Photo Retouching Service || Color correction || Background removal Service || image editing service

Stephen said...

I like your services. I really enjoyed reading your blog, you have worked very well on this website. If you want to know coffee maker Visit here

ebony lorange said...

Good topics and well blog shared. A lot of thanks for such useful shared. Image Retouching
Photo Editing Helps
Neck Joint
Photoshop Guideline
Clipping Path Tutorial
Drop Shadow Helps
Background Remove
Image Editing
Image Cut Out
Photo Manipulation

Koen Khastory said...

A very well-written and in depth blog. Keep working hard just like this.
Clipping Path Company

Charlie Jon said...

Very good post. It has some good learning points to it. Great job!

Paypal vs Payoneer for Freelancers