Fugitive Thought

In pursuit of easy Zentific development while on the go, I've been poking at my Asus EEE 901 for some time, trying to convince xen to work properly upon its somewhat unusual platform. Xen isn't necessarily appropriate for workstation virtualization (even though I make it work), so thus the trouble. I can't promise an experience like that encountered on a nice Opteron 8xxx or Xeon/i7 platform, but if you're like me this may be a good solution for you.

In fact, I'm posting from my eee running Xen this very moment.

As with all such things, the devil is in the details. The trouble with the EEE is its relatively recent hardware. The 2.6.20 kernel distributed by Asus with its custom xandros derivative is very hacked to support such hardware - not that this is a bad thing, because it works wonderfully, but it implies that stepping outside of that rather custom-built box introduces some fun. As it is, the wifi drivers are - as of 2.6.30 - still in the unofficial, not-endorsed, 'staging' section of the kernel, otherwise known as the scary experimental department. Yes, that's ten versions after the .20 kernel originally running this thing, which demonstrates just how new some of this hardware is from a driver perspective.

But I digress, as usual.

I've been running the netbook remix version of Ubuntu Hardy. This is essentially a normal Hardy install with a patched version of 2.6.24 to support hardware appropriately, plus a few userspace helpers and some nice interface tweaks for the tiny screen. At any rate, this distribution is entirely sufficient for Xen (I take issue with the Ubuntu Xen package maintainer and moreover the way Canonical is approaching virtualization, but that's a different post. :)

So, the details:

I've built Xen 3.4-rc4 and a 2.6.30-xen kernel directly derived from Jeremy Fitzhardinge's paravirt-ops tree at git.kernel.org. The former includes its own 2.6.18 kernel (with many backported drivers), but the system is only barely usable in that state. With my kernel, however "everything works" - in quotations because it's not really everything (the webcam doesn't for example), but that's good enough for me for the time being. If someone really cares, I'll poke at it some more.

The meat can be found at http://zentific.com/files/eee/pvops-dom0-xen-kernel.tar.gz and http://zentific.com/files/eee/xen-3.4-rc3-eee.tar.gz -- but be sure to peruse the readme I've put together for particulars. http://zentific.com/files/eee/readme

Hope this helps someone. If not, oh well. Fugitivethought is my personal braindump so I can forget about some of the details and come back to check later.

Any questions, notes, etc email me or leave a comment.

-s

Why yes, Zentific can do RDP.

and now, so can xen. :)

Email me or leave a comment if you are interested in the source..

This is a simple example you can re-use for splitting up processing of data across processes for faster execution. Put all of the data into the $set and fill in function process with what you want to do on the data, and let 'er loose! I'm personally using it for telnet scripts because the amount of time spent waiting for a single telnet session is horrible and I can run many sessions at once while I wait for the responses.

/**
 * Splits the given set into $count subsets that are of approximately equal size
 */
function array_split($set, $count) {
   $subset_size = ceil(count($set) / $count);
   return array_chunk($set, $subset_size);
}

/**
 * Forks into $process_count separate processes and executes the function
 * named in $job in each process to split up handling of the data in
 * $set across the processes.
 */
function fork_exec($set, $job, $process_count) {
   $subsets = array_split($set, $process_count);
   $children = array();

   // launch all of the children and store process list
   foreach ( $subsets as $a_set ) {
      $pid = pcntl_fork();

      if ( $pid == -1 ) die("Error forking");
      else if ( $pid == 0 ) { call_user_func($job, $a_set); exit(0); }
      else $children[] = $pid;
   }

   // wait for each process to end
   while ( count($children) > 0 ) {
      $pid = array_shift($children);
      pcntl_waitpid($pid, $status);
   }
}

// example set to work on
$set = array('a','b','c','d','e','f','g','h','i','j');

// Process the job with 3 threads and time it
$time = microtime(true);
fork_exec($set, 'process', 3);
$diff = microtime(true) - $time;
echo $diff . ' seconds for full run'."\n";

// This is the job to run on the set. Make sure it is multi-process safe!
function process($set) {
   foreach ( $set as $item ) {
      echo "Process [" . posix_getpid() . "] executing '" . $item . "'\n";
      sleep(1);
   }
}

Coming soon in this entry, details about hardening your xen installations.

Topics to be discussed include:

-paravirtual framebuffer: intended function, patch sets, and practical considerations. 3.0.4 and up.
-pygrub
-xenstore denial of service
-networking (ebtables, iptables, etc)
-vnc over SSL
-xenapi over SSL
-minios, stubdomains, pvgrub
-driver domains
-pci passthrough (why DMA spells disaster when lacking an IOMMU)

In progress and coming your way as time permits. Check back soon.

http://www.zentific.com :)

 

Forced backgrounding of processes

The following dissociates stdout/stderr/stdin (saves output in nohup.out) and removes controlling terminal so that you can close the terminal or log out while still permitting the program to execute.

nohup /path/to/some/command/that/isnt/a/daemon & disown %-

This can be useful for long operations that do not require interaction (otherwise you'd just use the screen utility of course).

For similar goals where this fails, justin has some code to solve the problem. Perhaps he will post it.

 

Shared terminal session, the ghetto method (readonly)

The following permits a user on terminal A to show another user on terminal B what transpires (in real time) during a terminal session.  A fifo is created as a first step and then script is run with its output directed to this fifo -- the script command will appear to hang, but when the user on terminalB uses cat upon the fifo, script will exit succesfully. This is somewhat useful - it alerts the user on termA that the other user is ready.

terminalA$ mkfifo /tmp/out
terminalA$ script -f /tmp/out

terminalB$ cat /tmp/out 

Now, you could use script to record a session and distribute the result afterward, but sometimes observing as things happen is useful. For a more flexible mechanism to accomplish the same, the GNU screen utility has shared session ability - including permissions with respect to reading/writing.

<djahandarie> we ain't here to do e-c-e
<djahandarie> we're here to do c-s-e on the w-e-b
<djahandarie> listen to me spit these rhymes
<djahandarie> while i program lines
<djahandarie> and commit web accessibility crimes
<djahandarie> word, son
<http402> You talk like your big on these I-Net kicks,
<http402> But your shit flows slower than a two-eighty-six.
<http402> I'm tracking down hosts and nmap scans,
<http402> While Code Igniter's got you wringing your hands.
<http402> Cut the crap rap,
<http402> Or I'll run ettercap,
<http402> Grab your AIM chat,
<http402> N' send a PC bitch-slap!
<http402> peace
<djahandarie> you're talkin bout down hosts and nmap scans
<djahandarie> while i got other plans
<djahandarie> you're at your new job, but you can't even do it right
<djahandarie> you just create a plight with your http rewrites
<djahandarie> i've been on the web since the age of three
<djahandarie> you just got on directly off the bus from mississippi
<djahandarie> respect yo' elders, bitch
<http402> You've been webbin' since three, but still ain't grown up,
<http402> Gotta update your config and send the brain a SIGHUP.
<http402> You say you're that old? No wonder you're slow!
<http402> You're knocking at the door while I run this show!
<http402> Elders my ass, you're shit's still in school,
<http402> Hunt and pecking at the keyboard like a spaghetti-damned fool,
<http402> Rim-riffing your hard drive like a tool,
<http402> Face it. I rule.
<djahandarie> i erase my harddrives with magnets (bitch)
<djahandarie> all you can do is troll on the fagnets
<djahandarie> and son, my brain's wrapped in a nohup
<djahandarie> it wont be hurt by the words you throwup
<djahandarie> dont mind me while i emerge my ownage
<djahandarie> while you're still over there apt-getting your porridge
<djahandarie> you say i'm still in school
<djahandarie> but the fact is that i know the rule
<djahandarie> cuz you need to go back to grade three
<djahandarie> and you better plea, that they take sucky graduates from c-s-e
<http402> Time to bend over and apply a patch,
<http402> Your brain's throwing static like a CD with a scratch.
<http402> Your connection got nuked and you've met your match.
<http402> You run a single process like a VAX with a batch.
<http402> I'd pass the torch to a real winner
<http402> But it'd just scorch a while-loop spinner
<http402> Caught in a loop that you cant escape,
<http402> I run clock cycles around your words and flows,
<http402> Cuz your rhyme is like a PS fan: it' blows,
<http402> Your water-cooled lyrics leak and it shows,
<http402> Take your ass back to alt.paid.for.windows.
<djahandarie> Good god, I can't even respond to that. :P
<djahandarie> You win haha
* http402 takes a bow