KROSS, the Kexi scripting bridge
started by: ccpasteur
extended by: sebsauer
status: slightly outdated as Kross2 exists
Introduction
Kross provides a common API to wrap functionality independend of the used scripting backend. That way we are able to use Python and KJS as scripting backends and are able to integrate easy more interpreters.
Kross is splitted into 3 parts;
- API
Used to have a common API for all supported scripting languages. It's designed as proxy for every kind of interaction and spends us the communication layer between wrappers and interpreters.
- Interpreter
The scripting backends. At the moment only Python is implemented, but KJS will follow. Well, even integration of KScript would be an idea :)
The interpreters don't know anything about wrappers.
- Wrapper
Wrappers are the bridges between Kross and the Application-/Library-functionality we want to be able to access from within interpreters.
The Wrapper doesn't know anything about the interpreter and therefore the scripting backend itself.
At the moment there exists a wrapper for KexiDB. The testcase shows an example where we use from within Python KexiDBDriver and KexiDBConnection to connect with a KexiDB database and walk with KexiDBCursor through a table to print the content of KexiDBField's.
See the Kross API Documentation.
^ toc
FAQ
^ toc
Why Kross?
As explained above the initial intention was to have an abstraction layer between Kexi and the used scripting backend. Not every user is motivated to start learning a new language to be able to script something. Some prefer one language over another. So, why we should cut down to one language if we are able to include with Kross different ones with the same rich set of functionality?
^ toc
Why not SIP/SWIG?
Because both of them arn't really that flexible and to rich for small parts like KexiDB. With Kross we are able to do anything we want to do without depending on the features and restrictions another application spends. The with Kross reached flexibility does allow us to open any door we even don't know about jet.
Also Kross is build on top of Qt/KDE and therefore we don't need to wrap first whole Qt and KDE to be able to wrap our own stuff. Kross handles all this with less code.
^ toc
KScript depends on dcop to marshall data and that would exclude any plattform where dcop doesn't run. Also all published classes needs to be inherited from QObject and at least KexiDB isn't. KScript uses Qt's signal and slot concept to implementate function-calls, but signals and slots arn't a good choose on accessing e.g. KexiDB functionality.
Also don't see Kross as replacement for KScript. It's "just" an abstraction layer between what we need to access and underlaying systems/interpreters. Well, in fact there is no reason to don't include KScript as a optional supported scripting backend on Kross. From my point of view this should be done to support the same interface hopefully a lot of KDE-Appliations will support one day. But it shouldn't be the one and only supported backend cause of the reasons named above.
^ toc
Licensing issue
Kross uses the Qt/KDE-framework and while Qt is licensed as GPL Kross needs to be licensed as LGPL. The reason is clear; think of a database-project/document that embeds scripting code and uses Kross to evaluate this code. It would be needed to license the scripting code and therefore the whole db-project under GPL or the user would need to buy a Qt-license. It's not only about companies that may like to keep there databases and documents closed, it's also about noncommercial users who don't like to publish there data under GPL just cause a little helper-script is included. They would need to comply with GPL if they are going to provide their db projects outside of there "homes".
Therefore we decided to don't provide Qt-functionality like QWidgetFactory through Kross per default (see also the kjsembed License Clarification).
But for sure Kross isn't only about scripting embedded into database-projects and specialy for scripting code that extends the functionality of an application it may be very usefull to be still able to access and use the power of Qt and maybe for the authors it isn't a problem to publish such scripts under GPL. So, what we like to do is to provide a way to let the author decide. That could be archived by providing an optional Qt Kross-plugin. If the plugin got used within the scripting code, the script then the needs to be GPL. If those plugin doesn't got used and therefore the scripting-code doesn't make use of e.g. Qt-widgets, then the author is free to do whatever he likes to do with the script. Think of it as a somewhat from the GPL linux-kernel vs. "tainted mode" inherited solution.
^ toc
Kexi depends on Kross?
No, no and never. I guess there arn't that much users around who really care if there is some functionality within Kexi they maybe don't use cause Kexi is just around on there system to e.g. create reports or edit content. On the other hand, why should such a bunch of code be included on such systems? My dream is to be able to decide at least on compile-time if Kross should be included at all and if, what parts (means what interpreters and which wrappers) should be. If someone knows that he doesn't need the KJS-Interpreter or doesn't need scripting at all, why we should bother him to have the code/functionality still included? The user decides! :-)
^ toc
How to try Kross out?
Kross is now located in koffice/libs/kross. Please see our advanced build notes how to download, compile and install Kexi with Kross scripting-support.
Kross, as in the codebase located in koffice/libs/kross, doesn't depend on anything beside Qt and KDE. If you like to just try it out, you are also able to just compile and run the testcases; 1) fetch the Kross sources from cvs and 2) make -f Makefile.cvs && ./configure --enable-scripting && cd libs/kross && make && su -c "make install" && cd test && make && ./krosstest
^ toc
How to use Kross in an application?
Kross::Api::ScriptManager is the common base to access Kross. Just call the static function Kross::Api::Manager::scriptManager() to get an instance to work with. Those static function will take care of sharing the manager-instance and freeing it if not used/referenced any more.
Kross::Api::Manager* manager = Kross::Api::Manager::scriptManager();
You are the able to add optional modules. Modules are external wrapper we have explicit to add to have them accessible at the scripting backend. Following line adds Kross::KexiDB to the manager.
manager->addModule(new Kross::KexiDB::KexiDBModule());
ScriptContainer are used to separate different scripts. Think of a ScriptContainer as one whole pythonfile. We create a new Kross::Api::ScriptContainer and fill it.
Kross::Api::ScriptContainer* scriptcontainer;
scriptcontainer = manager->getScriptContainer("MyScriptName");
scriptcontainer->setInterpreterName("python");
scriptcontainer->setCode("print \"Hello world python script.\"");
You are also able to publish whole QObject instances. Scripts are able then to connect signals and slots, execute slots, emit signals or get/set the Q_PROPERTY's the QObject spends.
scriptcontainer->addQObject(myQObject);
Once the ScriptContainer is filled at least with a valid interpretername and some scriptcode we are able to execute the whole ScriptContainer with
scriptcontainer->execute();
or connect signals from QObject instances with a scriptfunction.
scriptcontainer->connect(myQObject,
SIGNAL(mySignal(const QString&)),
"myPythonFunctionname");
or just call function in the ScriptContainer. With callFunction you could optional pass a Kross::Api::List instance as function-parameters.
scriptcontainer->callFunction("myPythonFunctionname");
^ toc
Integrate Kross with Kexi
The main purpose of Kross is to be integrated into Kexi as scripting backend. Following text tries to deal with the question how to connect both.
Please take a look at my current tasks to get a more detailed list of the progress on this work.
^ toc
Access Kexi from within Script
The best example for this situation is KexiDB itself. It should be possible (and in fact it is already) to access KexiDB from within scripting code and e.g. fetch the content of a table.
Also needed is a way to access in the script the actual form and read and change e.g. the content of a lineedit what could be done by implementating a wrapper around KexiFormDesigner. This could be archived by introducing a KexiFormDesigner-wrapper that uses Kross::Api::QtObject to wrap Qt signals, slots and Q_PROPERTY's.
We have to pass runtime-informations from Kexi itself to the script like commandline-parameters, actual opened forms, etc. too. A Kexi/KexiProject wrapper is needed to abstract the runtime informations and pass them to scripts.
With Kross::Api::QtObject we are already able to handle signals/events, but maybe it's needed to have a more generic solution like a signalmapper inside Kexi itself to catch e.g. the Kexi close event. Also passing/calling/creating of KAction's from within scripts may an idea. We also need some way to e.g. call the FileOpenDialog from within scripts and work with the returnvalue.
^ toc
Access Script from within Kexi
This is the most difficult part cause there are a lot of scenarios for that. Easiest case is the calling of script-functions on events like form1.button1.onClick. So, if someone clicks on a button a script should be executed.
^ toc
Kross spends Kross::Api::Manager and Kross::Api::ScriptContainer to access the functionality Kross spends from within other code. The ScriptingManager uses them to build a bridge into Kexi.
^ toc
Scripting editor
To let users edit and maintain scripting code Kexi spends the Scripting editor. While the editor spends Kross-specific functionality like debugging or executing scriptcode or to generate skeleton for a new function to handle signals, it should be still possible to compile Kexi with scripting editor but without Kross. That allows testing of the editor without the need to compile Kross.
^ toc
A GUI for defining function calls
This can be a component called Kexi Macros in the future.
E.g. see how it was done with Kommander: http://kde-apps.org/content/pre3/12865-3.png
^ toc
What else?
Let's spin the wheel to get at least some kind of overview in what cases we would or could need a connection between Kross and Kexi.
- Use Kross without Kexi
To name an example; wouldn't it be nice, if the Python-part to access KexiDB could be compiled as python-module too? That would allow e.g. to run a python-script from within Scribus, that uses KexiDB and the Scribus Python-API to generate a very nice Scribus document what could be used as report/export/whatever of the content we are able to access with KexiDB in a very flexible manner. The problem is, that each interpreter-backend could spend it's own flexibility here and that it's not that easy to compile a simple external python-module from within Kross. For sure it's possible, but would need a lot more Kross-abstraction.
- How to integrate Kexi core functionality like import/export/migration/driver?
Each of them would need there own kind of EventManager layer to keep independency between Kross and Kexi. Is it wise to try to keep them splitted by using #ifdef's all around or what would be the most flexible solution?
- oo2.org and MSA show us what is possible with scripts
They don't only use scripting as simple optional backend. They also use it to spend some parts of the core-functionality like Wizards, Import/Export and maybe even own kind of dialogs. While features like Import/Export are already handled at the point above, there stays still one big problem; interaction between the user and interpreters. The main and only problem I see here, is that it's needed to be able to create and show e.g. a QDialog or a Wizard from within scriptingcode totaly independend of the used interpreter and dynamic enough to build nearly a fully GUI-Dialog from within scripts. While it isn't that big problem to do that with KJS, Python's only way to create own Qt-Dialogs seems to be PyQt. Don't let us play with the idea to use Tk or something else each interpreter spends! The best solution would be, to write a wrapper, that will spend the same functionlity to each interpreter to create such QDialogs. If we take a look at Kommander it's possible and if we took a look at KFormDesigner we have already all the needed functionality around. The most powerfull solution would be to write a wrapper for KFD-like functionality (e.g. create a Form from UI-Code). That way we would be able to add different backends at the wrapper and e.g. use KFD, Kommander, Qt(-Designer?) or whatever without depending on one specialy solution/application.
- Replace Kross or add another Kross-like solution
Even if I don't hope that it will be the case, I guess we should keep in mind, that Kross may not the best / most powerfull solutions for next n years. Maybe some day I got enough free time and more experience to work on Kross2 or Trolltech starts to extend there QSA-solution to a LGPL Kross-like interpreter-layer. Whatever happens there next few years, we should try to design the integration flexible enough to be able to add or even replace the existing solution with another scripting-bridge. So, one more reason to try to work with flexible bridges between Kross and Kexi.
^ toc
Security issues
april 2005, js
In april 2005, Frank Karlitschek announced availability of kde-docs.org web site. With that we can upload (among others) .kexi database files (examples, templates, etc.). Security issues appeared quickly, at least in our minds.
>>>The question is how do we deal with is, will there be some kind of
>>>possibility to review what is uploaded? This is because at least Quanta
>>>has executable resources and you don't want to download a toolbar with
>>>a script which does a "rm -fR ~"...
>>
>>That's also my concern regarding incoming Kexi scripts, possible to execut
>>as autorun... We may need to introduce a process of validation for such
>>resources. Even "only trust well known authors" policy is not safe... Maybe
>>GPG signing?
>
> Yes. This is a problem. We can't distribute scripts at the moment. This would
> be a security problem for the users.
>
> Solution 1. A sandbox system for the scripts in the application. I think this
> doesn't exist at the moment and would be difficult to implement.
If we're talking about Kexi: we're curently supporting Python (javascript can be next) -- reusing restricted python interpreter is planned. We'll be very careful about releasing next (0.2?) Kexi version with scripting enabled. Unless scripting is enabled, it's safe to upload example .kexi database files.
Somebody could say the risk is the same as with possibility of uploding source code with unsafe/viral routines. The biggest difference is of course that documents/ databases containing scripts can hurt far easier just by clicking them.
Testing a file containing scripting by opening it even in a really powerful sandbox is not enough. Why? Imagine an attacker upladed a worm code which is activated after, say two days. Until next day, the file will be reviewed (because it's working great, has nice features, and so on) and then many folks will download and run it, in good will. Another day, and this reviewed piece of code becames a worm...
Following both suggestion try to deal with that issue.
^ toc
Security-Manager
The idea is, to have in Kross a common security-manager for all scripting interpreters as first wall. Let's imagine we would use some kind of gpg signing to have scripts categorized into trusted and untrusted. Trusted scripts would be just executed while untrusted ones are in the best case denied or, to be more flexible, the user is asked and warned about the danger. For sure this doesn't prevent users from quickly saying they trust an evil certificat, but at least spends us an easy way to limit possible damages on a first step.
I guess we could add a kexi/plugins/crypto plugin that spends us a kexi-wide possibility to encrypt/sign stuff. That way we could decide one day to introduce such a security-manager for other Kexi related functionality like forms too.
The crypto plugin could use Aegypten (like KMail, see kde/kdepim/libkdenetwork) to minimize the needed work and reuse a well maintained solution :) Does it work/compile under Windows too?
^ toc
Restricted python
Up to Python 2.4 it's not really possible to sandbox the interpreter. Till python 2.2.2, the bastion and rexec modules where used to restrict the embedded interpreter and minimize evil actions a script could start. Both modules are obsolete cause of ongoing security-flaws and official marked as "don't use". Till today python itself doesn't provide an alternate security-model, which is the biggest flaw python has imho.
Zope 3, the open source application server written in python, provides zope.security a very powerful python module to sandbox python in a restricted environment and define complex security-rules - see Untrusted interpreters. The problem is, that it's a very large codebase with a bunch of files and c-code.
The RestrictedPython module Zope spends does just sandbox the scripting code without all the power the zope.security framework spends. It's needed to wrap the objects to build an own security framework. I decided to go this way and don't use zope.security cause of following reasons;
- RestrictedPython has just a handfull of py-files. Therefore they are included in Kross and installed as python module.
- We don't depend on the complex Zope3 application server or needed to fork the zope.security framework.
- We've with RestrictedPython a very flexible solution in place to execute scripting code in a secure sandbox while the code to achieve it isn't that much.
^ toc
Sounds good, but...
We are very happy on any kind of feedback. Please feel free to contact us/me if you guess I missed something or if you like to get more informations.
|
