radioAe6rt

Some quick tips and tools in getting started with JXTA.

[tags]JXTA,p2p[/tags]

There is a lot of interest in private nets in JXTA, and understandably so. If you want to make money, history, or both with JXTA, one absolutely must come to terms with creating a private net in which your hosts can run without outside interference.

After some discussion with friends (thanks, Vanessa), we concluded that a simple standalone app that demonstrated private net creation and a working set of peers was in order.

Here is source code for creating a private net rendezvous and relay, and an application peer.

The code is not beautiful, or much commented, but if you’re into JXTA to any degree, you’ll see how it hangs together.

Unzip the zip file and modify build.properties to reflect where your JXTA distribution jars are.

Modify seeds.txt to reflect which host in your sandbox is the rendezvous/relay. As you can see, the bundled seeds.txt uses a typical home LAN type of network address.

Pick three hosts to run on. Call one of them the dedicated rendezvous/relay. On that host, run “ant rdv”. Logging output goes to ./rdvlog.txt. Tail the log file, looking for CLIENTCONNECT/CLIENTRECONNECT type of messages, just for fun and to convince yourself that your peers (below) are really connecting. If you never see a CLIENTCONNECT or CLIENTRECONNECT in the rdv log file, something is wrong. Check your host ACLs and such. Also, verify that an rdv listener comes up on port TCP 9701.

Pick two other hosts, copy the zip file to them, and run “ant node” on each. (Make sure the seeds.txt file is accurate!).

The “nodes” are real JXTA peers doing real work: they create a “propagate pipe” and write their PeerID and a random integer to it. The peers will not attempt to do this work, however, until they have connected to a rendezvous (see ‘waitforRdv()’ in MyPeer.java source). When the peer connects to a rendezvous, it will announce it to System.out.

Typical working (not including startup-type messages) System.out node output looks like this

 [java] (from other): urn:jxta:uuid-59616261646162614E504720503250338E1B00F1F7CC4F76BEBAE14EA96ED64403 says 369883458
 [java] (from self): urn:jxta:uuid-59616261646162614E50472050325033B75D6C9614D349F5AFCAF9A076718C7703 says -1309999697
 

Update 16 Nov 2006: I added “cache clearing” to both the build.xml Ant script and to the Rdv and Peer classes on startup.

Update 17 Nov 2006: Thanks to Jeff Schultz of the JXTA community for pinpointing a bug in the latest code. At this time the code runs for a lot of hours on end, and therefore constitutes a good working model on how to create a private p2p net with supported peers.

[tags]JXTA,p2p[/tags]

When you do JXTA development, sooner or later you will need new IDs for various purposes.

I wanted to learn how to write a Shell extension, so I implemented a new “mkids” command for the Shell.

Compile it, jar it, and add it to the Shell classpath when invoking the Shell:

#! /bin/sh
release=Eso
platform=$HOME/JXTA/Release
cp=$platform/$release/shell/binding/java/jnlp/lib/jxtashell.jar
cp=$cp:$HOME/projects/ShellExtensions/lib/shellextensions.jar
for i in $platform/$release/platform/binding/java/jnlp/lib/*.jar ; do
    cp=$cp:$i
done
set -x

# Set JXTA_HOME
home=$HOME/.jxta
# Clear the cache - a good habit!
rm -rf $home/cm

# invocation from http://download.jxta.org/build/release/latest/jxta-shell-2.3.7.zip
java -DJXTA_HOME=$home -classpath $cp net.jxta.impl.peergroup.Boot

To use it, run the Shell with the script above, and when the console comes up, type “mkids” to produce output that looks like this:

JXTA>mkids
Peer Group  ID  = urn:jxta:uuid-3C47FAB8ADCE46A6BEA79C8BE5337AB902
Peer ID         = urn:jxta:uuid-3C47FAB8ADCE46A6BEA79C8BE5337AB9D05AA5B90193498FBB55BA14DDD8FEBB03
Module Class ID = urn:jxta:uuid-BDDCD6F4FBE74A91B5E5A2A6C0876D7205
Module Spec  ID = urn:jxta:uuid-BDDCD6F4FBE74A91B5E5A2A6C0876D7257B26DC9438C45D4AEAA9C877F34F3F206
Pipe ID         = urn:jxta:uuid-3C47FAB8ADCE46A6BEA79C8BE5337AB9A25E8DF27E114C24AC9FBF5AE9B2C2AD04

[tags]JXTA,p2p[/tags]

Here is a quickly-hacked and very simple boilerplate for a JXTA rendezvous and relay that serves a private net.

Note that in configureJXTA() you must supply valid relay and rendezvous seeding URIs for “rdvSeedingURI” and “relaySeedingURI”. Those URIs must return the moral equivalent of something like this:

$ wget -O seed.out 'http://www.example.com/cgi-bin/jxtaseeds.cgi?q=rdv'

$ cat seed.out
tcp://192.168.0.4:9701
tcp://192.168.0.8:9701

You will also need the make.sh and run.sh scripts, as well as the log4j.properties file. The log config file must be in the runtime directory. Per the log4j.properties file, log output goes to ./rdvlog.txt.

Naturally, you’ll want to change the various URN-type IDs to values of your own choosing, but as long as you don’t share an IP address space with anyone else running this rendezvous, it shouldn’t really matter.

The NetPeerGroupFactory constructor may change in the future, so keep your eye on that.

[tags]JXTA[/tags]

Writing up in a bit more detail the recent work on Java app security profiling.

5 Nov 2006 update. I now believe this method is unworkable. See the updated tutorial preamble.

6 Nov 2006 update. On again, off again. I think I got it working. The joys of discovery.

21 Nov 2006 update: I added a simple cache to ProfilingSecurityManager to suppress the writing of duplicate rules during profiling. This saves a considerable amount of I/O.

[tags]java security, java security managers,tomcat,webapps[/tags]

This is an obscure topic, but integrating Tcl and C++ may be of interest to one or two people (solar system wide) out there. If you are working on vic, you’ll know what this all means, or at least why it’s discussed. Otherwise, probably not, and no harm.

Below are some experiences gained while integrating Microsoft DirectShow video capture into the venerable version of vic supported by University College London. References to source code below are to the CVS archive at UCL. The motivation for the work was to use a modern API for video capture under Windows, instead of the outdate VfW API UCL vic (and LBL vic, iirc) then sported. I took on the challenge of integrating DirectShow into vic after having been taunted by the prospect for at least a year. One day in late 2004, I’d had enough, and decided to plant my feet, dig in, and retire this problem.

As for learning DirectShow, I cannot recommend highly enough the book by Mark Pesce on the subject. I own hundreds of computer language and technology books, some good, some not so good. Pesce’s book presented me with exactly the information I needed in a way I could comprehend. Sometimes a book meets a problem and helps solve that problem in its entirety. This was one of those rare, but fortunate, occasions.

These are comments on how Tcl and C++ integrate into UCL vic version 1.1.5, for purposes of that DirectShow integration. Earlier versions likely follow the same scheme. UCL vic is found here.

Everyone’s learning style is different; everyone has a unique set of branch points in an inquiry, each of which consists of a required set of circumstance representing enough accrued understanding for the traveller to take on the next question. Getting there is a winding road, and this project was no exception.

Figuring out how vic gets from one point in its execution path to another took two weeks of uninterrupted work, in part because I had two weeks and in part because that’s how long it took. I enjoyed the work, and did not hesitate to stop along the way and really savor what was happening. I was not at the outset familiar in any meaningful sense with Tcl/Tk, and my C++ was rusty. Integrating DirectShow took one additional week of uninterrupted work.

Most of the early work consisted of doing recursive “find . -name \*.cpp -exec grep … ” or “find . -name \*.tcl -exec grep …” on the source tree to piece together the program flow. These “finds” were typically based on hints acquired from debug printf statements and string literals appearing in the various scripts of interest in vic/tcl in the distribution.

Part 1: UCL vic Tcl/Tk commands

Section 1: Tcl Base commands

The UCL distribution includes a Tcl/Tk interpreter that has “built-in” commands and to which can be added at run- or compile-time user-defined commands.

Built-in commands are defined in tcl-8.0/generic/tclBasic.c. These are the commands one normally reads about in commercial Tcl/Tk books, and will not be discussed further.

Application code can add user-defined commands to this base command set in one of two ways: by writing tcl scripts containing the familiar “proc” blocks, or by coding C++ classes that conform to the proper calling interface and adding them programmatically to the interpreter’s supported command set.

Section 2: Adding Tcl commands via .tcl script “proc” blocks

User-defined commands are added via tcl scripts through the action of tcl2c++ (vic/tcl2c++.c) on .tcl files. For each foo.tcl file, tcl2c++ produces a foo.cpp C++ source file, which in turn contains a *static* instance of class EmbeddedTcl. EmbeddedTcl classes contain a character-array representation of the input .tcl file, less the script file comments. The static keyword in the C++ source file causes the commands defined in the .tcl file to be created before main() runs, which is helpful to know. The Tcl interpreter then adds these EmbeddedTcl instances to its list of commands, alongside the existing list of built-in commands.

afaik, in UCL vic there is no significance to one Tcl “proc” command being defined in one disk file and another “proc” command being defined in another. The proc commands across all .tcl files reside in the same “name space”. So separate .tcl files are for the convenience of the programmer, and have no bearing on variable scope.

Section 3: Adding Tcl commands via external C++ code

A coder’s C++ class becomes an addition to the Tcl/Tk runtime built-in command set in one of three ways:

1. a static class can add a new command to the command set like the class VideoCommand does in vic/render/vw.cpp:

    static class VideoCommand : public TclObject {
       public:
         VideoCommand(const char* name) : TclObject(name) { }
       protected:
         int command(int argc, const char*const* argv);
    } video_cmd("video");
 

See the string literal “video” in the class definition. In tcl/ui-vdd.tcl, this “video” command is used near line 175 in proc “build”:

video $w $width $height

Therefore, when a string literal is seen functioning as a command in a .tcl file, and a corresponding “proc ” cannot be found in the application’s set of .tcl scripts, and the command is not a built-in Tcl command (e.g., “info”), grep through the C++ source for the string literal and examine the corresponding C++ input files.

2. vic adds its own user-defined “new” (”new” used as a string literal here) command to the Tcl interpreter’s base command set. See vic/Tcl.cpp for

    class CreateCommand : public TclObject {
       public:
         CreateCommand() : TclObject("new") {}
         int command(int argc, const char*const* argv);
    } cmd_create;
 

When the interpreter encounters the keyword “new” in a script, such as in vic/tcl/ui-ctrlmenu.tcl

set encoder [new module $fmt]

it, though the action of class CreateCommand above, walks an internal list of objects (”Matcher” class instances or descendants, for the curious) who have registered themselves under the string literal name “module” and queries each one to determine if it supports a particular $fmt. $fmt could be “h261″. In that case “[new module h261]” returns a pointer to a new instance of class H261PixelEncoder. As a result, “$encoder” is a pointer to an instance of the class H261PixelEncoder. And by “registered”, we mean transparently registered in a well-defined way in the application code itself, not in any abstract MS Windows “registry” or otherwise sense.

For example, and notice again the use of static classes for this matching class storage scheme,

    static class H261EncoderMatcher : public Matcher {
       public:
          H261EncoderMatcher() : Matcher("module") {}
          TclObject* match(const char* fmt) {
             if (strcasecmp(fmt, "h261/pixel") == 0)
                return (new H261PixelEncoder);
             if (strcasecmp(fmt, "h261/dct") == 0)
                return (new H261DCTEncoder);
             /* XXX for now, this is compatible with ui-ctrlmenu.tcl */
             if (strcasecmp(fmt, "h261") == 0)
                return (new H261PixelEncoder);
             return (0);
         }
    } encoder_matcher_h261;
 

This particular “helper” descendant of Matcher registered itself as type “module”, and knows how to return an instance serving H261 (”h261″) encoding. Again, this subclass of Matcher is on an internal list that the interpreter walks, looking for the properties module/h261 of interest.

3. user-defined commands can come into being by being returned as values of other user-defined commands (Ed.: In the end, Tcl commands are Tcl commands, built-in or user-defined - it’s just a question of what public facing mechanics brought them into being).

For example, in ui-ctrlmenu.tcl, a “grabber” command is created in $V(grabber) via

set V(grabber) [$videoDevice open $ff]

$V(grabber) is a pointer to a C++ class instance returned by the Tcl/Tk user-defined command $videoDevice, whose arguments are “open $ff”.

Section 4. User-defined command arguments

All user-defined commands must implement a public method conforming to

int command(int argc, const char*const* argv);

When the Tcl interpreter encounters a user-defined command o with arguments (or without), it will call this method on the object o in a o->command(argc, argv) fashion.

For example, the $grabber command responds to the tcl script command in vic/tcl/ui-ctrlmenu.tcl

$grabber decimate $inputSize

via its ::command() method in vic/video/grabber-win32DS.cpp

    int DirectShowGrabber::command(int argc, const char* const* argv) {

      if (argc == 3) {
          if (strcmp(argv[1], "decimate") == 0) {
             u_int dec = (u_int)atoi(argv[2]);
             Tcl& tcl = Tcl::instance();
             if (dec <= 0) {
                tcl.resultf("%s: divide by zero", argv[0]);
                return (TCL_ERROR);
             }
             if (dec != decimate_) {
                decimate_ = dec;
                if (running_) {
                   stop();
                   setsize();
                   start();
                } else
                   setsize();
             }
             return (TCL_OK);
          } else if (strcmp(argv[1], "port") == 0) {
             setport(argv[2]);
             return (TCL_OK);
          } else if (strcmp(argv[1], "useconfig") ==0) {
             if (strcmp(argv[2], "1") == 0)
                useconfig_=1;
             if (strcmp(argv[2], "0") == 0)
                useconfig_=0;
          }
       }
       return (Grabber::command(argc, argv));
    }
 

DirectShowGrabber descends from Grabber (vic/video/grabber.cpp), which descends from TclObject (vic/Tcl.cpp). TclObject allows descendants to override ::command() as it declares TclObject::command() as “virtual”. Descendants override ::command() to provide the specific behavior the descendant needs to implement the app.

Section 5: vic.exe main()

Finally, we can talk about what happens when main() in vic/main.cpp is called. There is not much to say here, except you can follow the tcl script control flow by noting the call to

tcl.evalc(”vic_main”);

proc vic_main is defined in vic/tcl/cf-main.tcl. With Section 1 above as a backgrounder, everything else sort of follows from this single call.

Part 2: DirectShow integration

The source for the DirectShow integration is in vic/video/grabber-win32DS.cpp. The original source for VFW is untouched in vic/video/grabber-win32.cpp — except for a #ifdef selecting which devices to instantiate at runtime, either the VFW devices or DirectShow devices.

Some key magic in capture device discovery happens before main() runs at the top of vic/video/grabber-win32DS.cpp with

 #ifndef VIDE0_FOR_WINDOWS
 static DirectShowScanner findDirectShowDevices;
 #endif
 

This single line does the system level capture device search and simultaneously registers each found device with the Tcl intepreter. As a result, $videoDeviceList in vic/tcl/ui-ctrlmenu.tcl contains meaningful pointers to real device class instances. Device names are acquired by the UI via

Tcl::instance().evalf(”lappend inputDeviceList %s”, name());

found in vic/device.cpp.

[tags]vic,videoconferencing,ucl vic,directshow,tcl and c++[/tags]