Alain Stalder
2014-12-14 08:00:56 UTC
Traditionally, you had it either "black" or "white", either you would
compile something before you ran it or you would compile (or interpret)
dynamically at runtime. In the first case, you would generally lose
flexibility, im the second case generally performance and control.
With "grey" I mean something somewhere in between and I think there
are many different ways in which that "in between" can be structured,
and I generally expect more and more of that in the future.
The Groovy SDK already addresses quite some cases, for example, the
GroovyShell does cache classes from compiling a script file, and so on,
but there are also cases, like the one I am about to present, where
the Groovy SDK, as it is today, ran into some design limitations.
But the following case story is also interesting independently of that,
mainly just a "real life" success story with Groovy...
A server-side Java application. Different users in different isolated
sessions perform some actions and some actions are predefined in some
sort of configured "workflow". These workflows can be configured in
different kinds of text files (like properties files, etc.).
For more flexibility in the workflow, it had been possible since
already a few years to use expressions in a simple script language
with a syntax similar to the expression language (EL) in JSPs,
namely in JEXL (an Apache Open Source project).
Some examples:
JEXL has lots of limitations, which have shown in practice. You can
run external scripts, but each script can only be a single function,
no classes, methods, functions etc. inside are possible. And many
simple things require strange workarounds, like calling static methods
of a class and more. And, for many simple things for which there are
libraries in modern script languages like Ruby, Python or Groovy,
specific functions had to be written, which, of course, required a
new software release each time.
So, the idea was to use a different script language. My idea was to
use Groovy, but others had to be considered too, mainly JRuby, Jython
and JavaScript.
Groovy won for mainly the following reasons:
- Language very similar to JEXL (like also JavaScript, but not the
others), this is important because the people who configure the
workflows are already familiar with JEXL but not necessarily with
any other script languages.
- Best integration with the JVM and language essentially a superset
of Java - especially important was that it would be possible to
implement temporary workarounds using almost Java code.
- This one may be a bit of a surprise: Groovy is more of a script
language for "programmers", so less chance that some adminstrator
who would be a great fan of Ruby or Python would start to create
big trees of scripts, maybe with many dependencies to external
libraries, which would in the end be hard to maintain.
Technically, integration was very simple at first. The variables
"user", "runscript", etc. in the examples above are just entries
in a Map which in case of JEXL had to be wrapped in a JEXL Context.
For Groovy, I could simply use the same Map instance to create a
Binding instance and thus automatically share variables between
JEXL and Groovy, for example like this:
where # marks a Groovy script expression. (Simply replacing JEXL
with Groovy was no option for backwards compatibility.)
The very first approach simply used the GroovyShell to run each
expression. Which means, that each script expression was compiled
once per invocation during a workflow, independently for each
user session. Which is, of course, not ideal for performance.
Ideally, you would compile such expressions only once and then
use the compiled result for each invocation. Static compilation
would have been difficult here, it was more than just properties
files, but also other text files that could be parametrized, and
the set of script files is intrinsically "grey" to some degree,
as in
so that in general, the set of scripts to compile is not known
at compile time of the Java application.
With the Groovy SDK, optimization was possible up to a certain
point. Per user session, it was possible to compile each script
expression and file only once, but the class files could not be
shared across sessions because then static variables could be
accessed between different sessions.
Imagine, for example, an administrator who writes a simple Groovy
script that wraps a username and a password in static variables
of the script class. These would become accessible in other
sessions and the administrator would not be able to predict this,
unless they would know about how Groovy scripts are implemented
in terms of the JVM.
With the Groovy JDK you have the option to define an output
directory for class files in the CompilerConfiguration and then
to create a new GroovyClassLoader based also on that directory.
But for various reasons, this approach was too limited for the
specific use case and instead I wrote Grengine.
With Grengine, what you cache is bytecode in memory, so that you
can then load classes for the same compiled bytecode in different
class loaders, so in the concrete case there was a separate
class loader per user session.
The implementation with Grengine also allowed to update scripts
while the application was running. Whenever a script file changes,
all new sessions automatically get new class loaders based on the
new scripts, but all existing sessions remain with the original
state. And, if compilation of the new scripts should fail, a new
user session would simply get a new class loader based on the
previously compiled scripts.
This would still not work 100% in all cases, but still enough to make
it practical to update scripts at runtime without fear for stability
and consistence of the application.
Some very brief general outlook:
In a Java VM, you generally expect the VM to optimize code at runtime.
Something similar would also be generally desirable for running Groovy
scripts: Ideally the Groovy JDK would learn and optimize compilations
to a minimum, more so and more dynamically so than already the case
today.
I hope that Grengine and this post have given a little bit of incentive
and inspiration into maybe implementing something like it some time.
As for Grengine itself, it is my impression that it is unlikely that
someone else will use it, also because I suck at "marketing".
Thanks for reading anyway. :)
Alain
--
View this message in context: http://groovy.329449.n5.nabble.com/Grey-Sets-of-Groovy-Scripts-A-Case-Story-with-Groovy-and-Grengine-tp5721936.html
Sent from the groovy - dev mailing list archive at Nabble.com.
---------------------------------------------------------------------
To unsubscribe from this list, please visit:
http://xircles.codehaus.org/manage_email
compile something before you ran it or you would compile (or interpret)
dynamically at runtime. In the first case, you would generally lose
flexibility, im the second case generally performance and control.
With "grey" I mean something somewhere in between and I think there
are many different ways in which that "in between" can be structured,
and I generally expect more and more of that in the future.
The Groovy SDK already addresses quite some cases, for example, the
GroovyShell does cache classes from compiling a script file, and so on,
but there are also cases, like the one I am about to present, where
the Groovy SDK, as it is today, ran into some design limitations.
But the following case story is also interesting independently of that,
mainly just a "real life" success story with Groovy...
A server-side Java application. Different users in different isolated
sessions perform some actions and some actions are predefined in some
sort of configured "workflow". These workflows can be configured in
different kinds of text files (like properties files, etc.).
For more flexibility in the workflow, it had been possible since
already a few years to use expressions in a simple script language
with a syntax similar to the expression language (EL) in JSPs,
namely in JEXL (an Apache Open Source project).
Some examples:
JEXL has lots of limitations, which have shown in practice. You can
run external scripts, but each script can only be a single function,
no classes, methods, functions etc. inside are possible. And many
simple things require strange workarounds, like calling static methods
of a class and more. And, for many simple things for which there are
libraries in modern script languages like Ruby, Python or Groovy,
specific functions had to be written, which, of course, required a
new software release each time.
So, the idea was to use a different script language. My idea was to
use Groovy, but others had to be considered too, mainly JRuby, Jython
and JavaScript.
Groovy won for mainly the following reasons:
- Language very similar to JEXL (like also JavaScript, but not the
others), this is important because the people who configure the
workflows are already familiar with JEXL but not necessarily with
any other script languages.
- Best integration with the JVM and language essentially a superset
of Java - especially important was that it would be possible to
implement temporary workarounds using almost Java code.
- This one may be a bit of a surprise: Groovy is more of a script
language for "programmers", so less chance that some adminstrator
who would be a great fan of Ruby or Python would start to create
big trees of scripts, maybe with many dependencies to external
libraries, which would in the end be hard to maintain.
Technically, integration was very simple at first. The variables
"user", "runscript", etc. in the examples above are just entries
in a Map which in case of JEXL had to be wrapped in a JEXL Context.
For Groovy, I could simply use the same Map instance to create a
Binding instance and thus automatically share variables between
JEXL and Groovy, for example like this:
where # marks a Groovy script expression. (Simply replacing JEXL
with Groovy was no option for backwards compatibility.)
The very first approach simply used the GroovyShell to run each
expression. Which means, that each script expression was compiled
once per invocation during a workflow, independently for each
user session. Which is, of course, not ideal for performance.
Ideally, you would compile such expressions only once and then
use the compiled result for each invocation. Static compilation
would have been difficult here, it was more than just properties
files, but also other text files that could be parametrized, and
the set of script files is intrinsically "grey" to some degree,
as in
so that in general, the set of scripts to compile is not known
at compile time of the Java application.
With the Groovy SDK, optimization was possible up to a certain
point. Per user session, it was possible to compile each script
expression and file only once, but the class files could not be
shared across sessions because then static variables could be
accessed between different sessions.
Imagine, for example, an administrator who writes a simple Groovy
script that wraps a username and a password in static variables
of the script class. These would become accessible in other
sessions and the administrator would not be able to predict this,
unless they would know about how Groovy scripts are implemented
in terms of the JVM.
With the Groovy JDK you have the option to define an output
directory for class files in the CompilerConfiguration and then
to create a new GroovyClassLoader based also on that directory.
But for various reasons, this approach was too limited for the
specific use case and instead I wrote Grengine.
With Grengine, what you cache is bytecode in memory, so that you
can then load classes for the same compiled bytecode in different
class loaders, so in the concrete case there was a separate
class loader per user session.
The implementation with Grengine also allowed to update scripts
while the application was running. Whenever a script file changes,
all new sessions automatically get new class loaders based on the
new scripts, but all existing sessions remain with the original
state. And, if compilation of the new scripts should fail, a new
user session would simply get a new class loader based on the
previously compiled scripts.
This would still not work 100% in all cases, but still enough to make
it practical to update scripts at runtime without fear for stability
and consistence of the application.
Some very brief general outlook:
In a Java VM, you generally expect the VM to optimize code at runtime.
Something similar would also be generally desirable for running Groovy
scripts: Ideally the Groovy JDK would learn and optimize compilations
to a minimum, more so and more dynamically so than already the case
today.
I hope that Grengine and this post have given a little bit of incentive
and inspiration into maybe implementing something like it some time.
As for Grengine itself, it is my impression that it is unlikely that
someone else will use it, also because I suck at "marketing".
Thanks for reading anyway. :)
Alain
--
View this message in context: http://groovy.329449.n5.nabble.com/Grey-Sets-of-Groovy-Scripts-A-Case-Story-with-Groovy-and-Grengine-tp5721936.html
Sent from the groovy - dev mailing list archive at Nabble.com.
---------------------------------------------------------------------
To unsubscribe from this list, please visit:
http://xircles.codehaus.org/manage_email