[1] where business rules can be stored as text without the application having to be recompiled. We’ll be covering all of these use cases but starting with one of the simpler use cases: creating an executable that launches a Python application.

[2]

[3]

Once we have set up a project that references these assemblies, we can extract a Python language engine from their grasp with the following snippet of C#:

using IronPython.Hosting;
ScriptEngine engine = Python.CreateEngine();

The VB.NET equivalent is

Imports IronPython.Hosting
Dim engine As ScriptEngine = Python.CreateEngine()

chapter 15 folder of the downloadable sources, as Visual Studio projects in both C# and VB.NET. Because of space restrictions we only show code examples in either C# or VB.NET, except where there are notable differences between them.

This code creates a single engine. The Dynamic Language Runtime supports the creation of multiple runtimes within a single application. This can be extremely useful for creating execution contexts that are isolated from each other,[4] a feature that Resolver One uses to have multiple open but separate spreadsheets.

In the examples that follow we will cover the most common ways of working with the IronPython hosting API, but you often have a choice of several different routes to achieve the same end. For more complete documentation on all the possibilities, you [5]

Once we have created an engine, we are ready to execute code, either from a string or directly from a file. At this stage our goal is to create an executable that launches a Python application, so we want to run a file.

Figure 15.1 shows a diagram of the DLR hosting API that we’ve used to get to this point.

Listing 15.1 extracts the sys module and sets the command-line arguments.

table 15.1.

Listing 15.2 shows code that creates a List (a generic .NET List, not a Python one) and populates it with the directory containing the Python program plus any paths from IRONPYTHONPATH. It then calls engine.SetSearchPaths with an array from the List.

Listing 15.3 does the same thing, but in VB.NET.

[6] (an integer), and if this happens we want to propagate the exit code. Instead of Execute, we can use ExecuteProgram, which returns us the integer exit code (and will create its own scope rather than requiring us to pass one in). Listing 15.4 executes the ScriptSource we created from program.py inside some exception-handling code. If the program terminates normally, it exits with the return code from ExecuteProgram. If an exception is raised, it writes out the formatted exception message and exits with a return code of 1.

figure 15.2.

[7] which covers the core classes and basic techniques for embedding IronPython. In the next section we build on this with a more specific example that uses IronPython to add a plugin mechanism to a program.

Topics we cover in this section are creating compiled code objects from Python code, setting and fetching variables from execution scopes, adding references to [8] This permits some very interesting interoperability stories. Where IronRuby (or Managed JScript or IronScheme or ...) uses IronPython objects, they retain their behavior as Python objects but are still usable from these other languages. There are some restrictions in the ways they can interoperate; it is unlikely that Python classes will ever be able to inherit from Ruby classes or vice versa, for example.[9] It should still be possible for dynamic languages running on .NET to share libraries, though. Python on Rails or Ruby on Django, anyone?

From a hosting point of view, the interesting thing we can do with scopes is to set variables in and fetch variables out of them. This means that you can publish an object table 15.2.

[10]

Fetching variables has more complications associated with it, mainly because of the impedance mismatch of interacting with a dynamic language from a statically typed language. So long as the variable exists—and after executing arbitrary Python code there’s no guarantee of that—C# and VB.NET insist on knowing the type before they will allow you to do anything useful with it.

For known types you have a choice of checking that the variable exists with ContainsName and use the generic version of GetVariable to fetch it from the scope. To fetch a string you use GetVariable(name) from C# or GetVariable(OfString)(name) from VB.NET. Alternatively you can use TryGetVariable, which takes an out parameter that will be null (nothing) after the call if the variable doesn’t exist.[11] From C# you will then need to cast the value to the known type after fetching it. TryGetVariable returns a Boolean indicating success or failure of attempting to fetch the variable.

If you are executing arbitrary code you will, therefore, need code that can handle the variable not existing or being the wrong type. If you are executing known code rather than arbitrary code, then it is fine to do any necessary error handling within the Python code and be able to guarantee that the variable exists and is of the expected type.

For .NET types, once you’ve pulled them out of the scope you have full access to them as if they were created from C#/VB.NET. If they are dynamic objects, such as functions or classes, you can still use them—but you have to use mechanisms provided by the Dynamic Language Runtime to perform operations on them.[12] This is something that we will look at later in the chapter.

In the meantime we now have all the pieces we need to set variables in a scope, execute code in that scope, and then fetch objects back out. Listing 15.5 is the start of an Engine class in C#. You instantiate it with Python source code as a string.

[13] The advantage of using CreateScriptSourceFromFile is that it doesn’t read the whole file at once. If your top-level program depends on the name being set to __main__, then you can have the best of both worlds by using the three-argument form: engine.CreateScriptSourceFromFile(path, Encoding.Default, SourceCodeKind.Statements).

This allows you to execute the ScriptSource in a scope with an explicit __name__ set, without it being implicitly overridden because it is from a file.

[14]

_runtime.LoadAssembly(typeof(String).Assembly);
_runtime.LoadAssembly(typeof(Uri).Assembly);

Which translates to this in VB.NET:

_runtime.LoadAssembly(GetType(String).Assembly)
_runtime.LoadAssembly(GetType(Uri).Assembly)

The BasicEmbedding example also includes a ClassLibrary.dll assembly containing a class with a couple of static methods (shared functions in VB.NET–speak) that write to stdout. We can add a reference to this assembly by first loading it with Assembly.LoadFile from the same directory as the executable. This time we have the example code in VB.NET, shown in listing 15.6.

[15]

The following snippet of VB.NET creates a ScriptScope called inner in which we set a string with the name HelloWorld. This is wrapped in a Scope object that is then published into the runtime globals.

Dim _module As Scope
Dim inner As ScriptScope

inner = _engine.CreateScope()
inner.SetVariable("HelloWorld", "Some string...")

_module = HostingHelpers.GetScope(inner)
_runtime.Globals.SetVariable("Example", _module)

Code running in the embedded engine can either execute import Example and access HelloWorld as a module attribute or execute from Example import HelloWorld to get direct access to the string we set in inner.

We’ve now covered all the major classes necessary for a wide range of different embedding scenarios. Figure 15.3 summarizes what we learned so far. It shows the core classes that we have worked with and the relationship between them and their useful members. Our core Engine class is now basically complete, with only one minor modification needed. Since we know how to make modules available for importing, we turn the scope in which we execute the main script into a proper module.

[16]

This C# snippet does this and puts the module into the runtime globals with the name __main__:

_scope = _engine.CreateScope();
_scope.SetVariable("__name__", "__main__");

Scope _main = HostingHelpers.GetScope(_scope);
_runtime.Globals.SetVariable("__main__", _main);

[17]

Adding a Python source file as an embedded resource to a Visual Studio project is as simple as adding the file (either from an existing file or adding a new text file and renaming) and setting the Build Action to Embedded Resource, as shown in figure 15.4.

Listing 15.7 shows the C# code to retrieve the source code from the embedded resource as a string.

Listing 15.8 shows the full code in VB.NET.

figure 15.5.

[18]

We also have to provide a mechanism for the plugins to be added to the application. We can do this with a PluginStore class, also accessible to user code, which acts as a registry for plugins.

Our PluginBase class is instantiated with a name, which will be used for the toolbar button. It also provides an Execute method, which does nothing on the base class but will be called with the textbox in real plugins (when the corresponding toolbar button is clicked).

On the .NET side, where we need to interact with the user plugins, we can use the PluginBase type. Listing 15.9 shows an implementation in C#.

Listing 15.10 shows the PluginStore class in VB.NET.

Listing 15.11 shows the Python code that creates a new plugin and adds it to the PluginStore.

Listing 15.12 shows the code that loads and executes all the plugins. Any errors in executing plugin code are caught, and a message box is displayed to the user, but syntax errors and SystemExit exceptions[19] are treated differently with a custom message. Figure 15.6 shows the error message shown to users for syntax errors.

Listing 15.12 is the C# code that loads and executes the scripts from the plugins directory.

[20] is through runtime.IO.SetOutput and runtime.IO.SetError methods that take a .NET Stream and an encoding. To use these methods, we need a Stream that diverts everything written to it back to the textbox.

Stream is an abstract class—inheriting from it is easy but requires implementing a tedious number of methods and properties. For this example we’ve chosen to inherit from MemoryStream and override the Write method. This is an abuse of MemoryStream, but it works fine.

Listing 15.13 shows the PythonStream class and setting an instance onto the runtime to divert both standard output and standard error.

figure 15.5 actually come from print statements in the plugin code. When the plugin is loaded (executed), anything written to standard out appears in the main textbox.

The final thing we need to do is to create a toolbar button per plugin and hook up its Click event to call the appropriate Execute method.

listing 15.14.

listing 15.15, has to be slightly different.

[21]

So far, whenever we have executed Python code from a string we have passed in the enumeration member SourceCodeKind.Statements. This member has a sister, SourceCodeKind.Expression, that allows us to evaluate an expression and return an object. It is used in this snippet of C# to evaluate a simple mathematical expression:

string code = "2 + 3 + 5";
ScriptSource source;
source = runtime.CreateScriptSourceFromString(code,
   SourceCodeKind.Expression);
int result = source.Execute(scope);

[22] as well: InteractiveCode. This can detect incomplete statements and is useful for executing code from interactive sessions, from a TextBox acting as a console in a UI, for example. You can call ScriptSource.GetCodeProperties(), which returns a SourceCodeProperties value that will tell you if the source code is invalid or incomplete.

Since we are retrieving results with specific types, we can use the System.Func delegate to define functions in Python and call them from other .NET languages. This delegate is a standard part of .NET 3.5 (C# 3.0), but it is also provided by IronPython 2, so you can happily use it in .NET 2.0 projects.

The simplest example of this is to evaluate an expression that returns us a lambda function, as shown in listing 15.16.

Listing 15.17 shows the equivalent VB.NET code.

[23] Figure 15.7 shows some of the members on the Python List type.

Listing 15.18 shows creating a tuple in Python and then using it from C#.

[24] This is because Python classes can have members added and removed at runtime. You can even change their base classes dynamically, things that we couldn’t do with .NET classes.

Python classes are still usable from C# and VB.NET; we can actually solve the type problem by ignoring it! We can keep objects as objects on the .NET side and use the Python engine to perform operations like creating instances and calling methods.

The mechanism for doing this is to use ObjectOperations, a class that provides dynamic operations for DLR objects. The ScriptEngine exposes this as the Operations property, returning an ObjectOperations instance bound to the semantics of the engine’s language.

ObjectOperations is a class with many useful methods, and again the best way to find out what it can do is to explore it with Reflector or the Visual Studio Object Browser. It knows how to perform comparisons and mathematical operations on dynamic objects, following the semantics of the language for those operations. In the case of Python that means autopromoting integers to longs if necessary and using the __add__ method for addition where appropriate, and so on.

More important, ObjectOperations knows how to call objects and set and fetch members. Using these capabilities alone we can achieve most of what we might want to do with dynamic objects. Listing 15.19 shows how to fetch a Python class out of an execution scope, create an instance of that class, and call a method on it.

Table 15.3 lists methods of ObjectOperations, including the three we have already discussed.

Listing 15.20 shows an example of using both the isinstance and the issubclass functions from C#.

Listing 15.21 uses the Python pickle module[25] to serialize and deserialize a Python dictionary. It then checks that the serialization and deserialization have worked, using [26]

[27] that the DLR would be integrated into version 4.0 of the .NET framework. Alongside this there will be changes to C# and Visual Basic to introduce dynamic features that use the DLR.

The major change in C# 4.0, at least the one that is relevant to us, is the addition of the dynamic keyword. This is a static declaration to the compiler that operations on the object are to be handled dynamically at runtime! Operations on objects declared as dynamic will be delegated to the DLR. For ordinary .NET objects the DLR uses reflection (just as it does inside IronPython), but it also enables some things not normally possible from C#. These include duck typing, the use of late-bound COM, and the creation of fluent APIs,[28] like XML or DOM traversal using element names as object attributes.

More important, it allows you to receive objects from a DLR language engine and use them as dynamic objects. Objects created by IronPython or IronRuby can be used from C# while retaining their behavior as Python and Ruby objects.

The C# Future documentation[29] gives this example of using the new dynamic keyword. All of the uses of d shown here will be done by the DLR:

dynamic d = GetDynamicObject(...);
d.M(7); // calling methods
d.f = d.P; // getting and settings fields and properties
d["one"] = d["two"]; // getting and setting through indexers
int i = d + 3; // calling operators
string s = d(5,7); // invoking as a delegate
int a = d; // assignment conversion

The final operation creates a typed object (a) from the dynamic object d. As with the examples you’ve been working on in this chapter, these operations could raise runtime errors, and so the sort of error handling that we’ve been discussing will still be needed.

The bottom line is that the hosting APIs make it easy to work with dynamic languages, but interacting with dynamic objects will get a whole lot easier.

[30] get easier, but there are many ways you can minimize the intricacy. One useful principle is to do as much as possible inside the engine. If you do your type checking and error handling in Python, then you can guarantee to return a known type into your statically typed code. When you really want to work with a dynamic object, then ObjectOperations is your friend.

We’ve now used IronPython from both the inside and the outside, and we’ve also reached the end of the last chapter. This willingness to experiment, and an excitement about the possibilities, is the most important message of the book. Have fun programming!

---

[1] Domain Specific Languages—These are “little languages” that encapsulate rules for a specific problem domain.

[2] See http://www.ironpythoninaction.com/.

[3] The IronRuby project has a similar entry point with easy access to engines and runtimes preconfigured for working with IronRuby.

[4] There is also support for creating separate runtimes in different AppDomains for further isolation.

[5] This is available from the DLR project page on CodePlex: http://www.codeplex.com/dlr.

[6] A Python program exits with an explicit return code by calling sys.exit(integer).

[7] We use the BasicEmbedding example in the downloadable sources.

[8] The Silverlight DLRConsole application does exactly this.

[9] If they could, which metaclass should they use—the Python one or the Ruby one?

[10] This mirrors the behavior of Python, where you can also programmatically set invalid identifiers in a namespace.

[11] Or if the variable is set to None inside the scope.

[12] This is not hard, but will be even easier once support for dynamic operations is built into the CLR or C#/VB.NET languages—as is happening in C# 4.0 and VB.NET 10.

[13] This is the same behavior when importing a module in Python; the name in the module corresponding to the file os.py is os, for example.

[14] The choice of Uri is entirely arbitrary. We just need some class that lives in System.dll.

[15] HostingHelpers lives in the Microsoft.Scripting.Hosting.Providers namespace. ScriptScopes are “remotable wrappers” for Scopes. HostingHelpers.GetScope gets the local version, so it will work only for local ScriptScopes. There are other ways of creating Scopes when using the DLR remoting support.

[16] Some of these classes have other useful members. This diagram is a reference to the ones we have used so far.

[17] Although you could encrypt your Python source code. Because that will require the means to decrypt the code in the assembly as well, it is still discoverable—but probably more work than assemblies compiled from C#, which are usually trivially disassembled with Reflector.

[18] Additional user-defined methods and members won’t be directly visible, of course.

[19] This is raised if the user calls sys.exit(n).

[20] We could also do this from within Python code by replacing sys.stdout and sys.stderr with custom objects.

[21] You can see an example of embedded IronPython as a calculator at http://www.voidspace.org.uk/ironpython/dlr_hosting.shtml.

[22] Actually it has two more. The purpose of SourceCodeKind.SingleStatement speaks for itself.

[23] This extremely useful tool for introspecting .NET assemblies is available at http://www.red-gate.com/products/reflector/.

[24] All Python classes are instances of their metaclass, and for new style classes the base metaclass is type. type can also be used as a function to tell you the type of objects.

[25] See http://docs.python.org/lib/module-pickle.html.

[26] CodeContext lives in the Microsoft.Scripting.Runtime namespace.

[27] Jim’s blog has links to videos of their talks: http://blogs.msdn.com/hugunin/archive/2008/10/29/dynamic-language-runtime-talk-at-pdc.aspx.

[28] .NET objects that implement the IDynamicObject interface can provide custom behavior when used dynamically.

[29] This documentation is available from http://code.msdn.microsoft.com/csharpfuture/.

[30] For example, see the discussion about C# 4.0 at http://channel9.msdn.com/posts/Charles/C-40-Meet-the-Design-Team/.