unflyingobject

There's no built-in for checking if a file exists when developing Dashboard Widgets in JavaScript. But with a little help from the BSD subsystem we can do it easily. For this to work, you obviously need to have

<key>AllowSystem</key>

<true/>

set in your Info.plist. That opens up a whole world of possibilities. I find that the most effective way is to use file. This way we can even check if something is of the correct type. For example, for a recent UnitInfo update, I had to check if InCrease was installed:

foldingBase = "~/Library/InCrease/cpu1/"; 

var fp = widget.system ('/usr/bin/file -bi ' + foldingBase + 'unitinfo.txt', null).outputString.replace ("\n", "");



if (fp.indexOf ("text/plain") != -1)

{

  clientVersion = 'InCrease';

} else {

  foldingBase = "~/Library/Folding@Home/";

  clientVersion = 'Folding@Home';

}

   

unitInfoFile = foldingBase + 'unitinfo.txt';

clientConfigFile = foldingBase + 'client.cfg';

I've been a big fan of Camino. Ever since it was called Chimera. Whenever a new Mac OS would come out, that I didn't have the hardware for, I could still use a modern, native Mac browser thanks to them. These guys have done a tremendous job bridging the Gecko rendering engine with a OS X L&F that in many ways feels more "Mac" than Apple's own browser. I originally switched to Camino because:

• 
  
• Many JavaScript WYSIWYG HTML editors will just disregard Safari. And if you have to use two browsers sometimes, why not just completely switch to the one that does everything, right?
• 
  
• It tries to render XML instead of showing me the tree
• 
  
• Safari has a weird habit of tagging .txt to downloaded files it can't recognize
• 
  
• Source view is very primitive with no highlighting and it also can't be reloaded.
• 
  
• Bookmarks menu has to be one of the slowest in the world
• 
  
• Downloads can't be opened with a double-click in the Downloads window. EDIT: Actually they can. Just click on the icon.
• 
  

I liked Camino because it eliminated most of my biggest problems with Safari, but now I'm forced to swtich back because:

• 
  
• Flash performance in Camino is just terrible. I tried to go around this by disabling it altogether, but in this YouTube day and age it's very difficult to do. And why should you? This got incredibly annoying at times - not being able to swtich tabs when a flash object was loaded in the frontmost window or having the whole app freeze for tens of seconds. Any kind of Flash content would also heaviliy bump Camino's CPU usage.
• 
  
• Other plugins behaved oddly as well. Often an embedded QuickTime movie would stay inside every tab I would switch to. Clearly something's not right with the Gecko View.
• 
  
• Window resize performance. I typically have about 10 tabs open at once and window resizing gets really slow. It's not disastrous, but definitely not something you would expect on a 2-year old Mac with 1,25GB RAM.
• 
  
• Stability issues. Especially in the latest 1.1 branch which would regularly crash every day for me. Downloading the latest nightly seemed to help though.
• 
  
• No AcidSearch. You can add search sites to Camino by editing an XML file but that won't give you the shorcuts that you have in Safari. This is clearly something that could be added to Camino but I personally have neither the skill nor the time and motivation to do this right now.
• 
  
• Startup time. Clearly no biggie, but in this day and age you would expect your browse to come up instantly. Almost so with Safari, but not quite with Camino. Here're some completely bogus test results, Camino takes more like 5 secs to start up actually (open returns before all the NIBs are unpacked, I would guess):
• 
  

> time open -a Camino

real    0m0.504s

user    0m0.080s

sys     0m0.073s



> time open -a Safari

real    0m0.272s

user    0m0.079s

sys     0m0.051s

In general, I'm not quite sure what to think of Camino anymore. Firefox is getting more and more Mac-like (to some extent) so maybe it would make sense for the two projects to just merge? That Camino would just be the Mac port of Firefox? Camino will run on anything starting with Jaguar which might be (?) a reason it's still so rough around the edges. It is my understanding that backwards compaitility often comes at a high price on OS X (although maybe not so much in Camino's case since it probably doesn't depend on OS services that much).

In any case, I replaced Camino with Safari in my Dock today. Bookmarks would've been trivial but let's be honest - who uses them anymore? del.icio.us, maybe. The biggest nag will actually be all the cookies that have already been set in Camino and the keychain items...

I had known about this before, but never really tried them until yesterday. I needed a really quick way to post photos from my phone, so decided to write a little photogallery app. You just dump pictures off the phone to a certain directory and that's it. In the process of reading about imagecopyresized, I stumbled on this method of embedding images:

<img src='data:image/jpg;base64,/9j/4AAQSkZJRgABAQAAAQABAAD' alt='Picture' />

That's called a data URL and they were first introduced in 1998.

In PHP, I couldn't find a way to read the data directly with imagejpeg(), so I end up overwriting 1 thumbnail for a whole directory of images. Working with images, the PHP memory limitations will quickly come into play. Anyways, here's the whole PHP code that goes through a directory and creates 80x80 px thumnails of a random portion of the picture. You can also see it in action. The source for that script is here.

$id = 'data/';

$dh = opendir($id);



while ($f = readdir($dh))

{

  $fp = $id . $f;

  if (is_file($fp) && $f != 'tmp')

  {

    list($w, $h) = getimagesize($fp);

    $scale = 0.6;

    $ns = 80;

    $im = imagecreatetruecolor($ns, $ns);

    imagecopyresized($im, imagecreatefromjpeg($fp), 0, 0, 

      rand(0, $w*$scale), rand(0, $h*$scale), $w*$scale, $h*$scale, $w, $h);

    imagejpeg($im, 'data/tmp');

    echo "

<a href='javascript:show(\"$fp\",$w,$h)' title='" . date('r', filectime($fp)) . "'>

   <img src='data:image/jpg;base64," . base64_encode(file_get_contents('data/tmp')) . "' />

</a>";

  }

}

I find it ironic, that things get increasingly difficult to find as their physical proximity increases (ie distance decreases). Thanks to Google, I can easily find a solution to a problem some guy, thousands of kilometres, and more importantly, years away, found and yet I'm utterly helpless if I have to find my bus ticket under the pile of papers that is my workdesk. In other words, filipp's search theorem states that "the physical distance of an object is inversely proportionate to the chance of me actually finding it".

My media library is another example (and I'm sure I'm not alone in this). I have tends of thousands of files stored on different mediums (computers and discs) within the confines of my home, only tens of meters apart, but when I need to find a single file, it's pretty much hopeless. While I can't really do much about my workdesk - it's just a force of habit, I'm still convinced I can do something about those files.

So what are the solutions?

1) Consolidate your storage by getting a bigger disk. I could image this working quite well. If I had about a terabyte of local storage, I could just store everything in one place, and use Spotlight to find it. Ofcourse, this is a band-aid. In a year, would need 2 TB, at least. It's also expensive.

2) Stop hoarding stuff. This isn't really an option for me, but the basic idea is that the really important data doesn't really take up much space. For me it's mostly text, with some pictures that I could easily fit within 10 GB. The rest (music and videos) is just enterntainment, right? Applications have become the most disposable form of "hd filler". But then you also have music bought from iTunes that you simply have nowhere else.

3) Start using some sort of DAM (Digital Asset Management) system. We're already doing this in many areas - Mail.app, iTunes, iPhoto, del.icio.us etc are all managing some assets. But which one should I install? I'm sure there's a really fancy and expensive solution out there, but all I really need is a generic tool that would essentially take snapshots of folders and then allow me to search them.

4) Hack Spotlight to support read-only media and file servers. I actually tried this, but after failed attempts to index SMB shares, I quickly gave up. Who knows, Leopard might help here, but I'm not betting on it.

Taking a snapshot of a folder on a UNIX system, is ofcourse a piece of cake:

> find ~/Movies \! -name '.*' > movies.txt

With more effort this could be made to check for duplicates (when importing the same folder again) and all sorts of cool things. Things will get hairy when I ned to store some extra attributes for an item. Like the QuickTime comment of a .mov or the year of an MP3. For this, a flat file becomes tedious.

The snapshot approach is really all I need - a medium-agnostic method for indexing and offline-searching file archives. So I'm starting yet another sideproject - a Python script that does exactly that. Here's what I have so far:

> ./lumi.py /Volumes/300G/Movies/

 > importing /Volumes/300G/Movies/

 > Successfully imported 1296 items

> ./lumi.py list

 > 2007-03-17T12:22:23.827078 (/Volumes/300G/Movies/)

 > 2007-03-17T12:23:04.715256 (/Volumes/Shared Items/Movies/)

> ./lumi.py name blender

 > /Volumes/300G/Movies/Training (blendervt-interface-3dviewport-v234-r0.avi)

 > /Volumes/300G/Movies/Training (blendervt-interface-concept-v234-r0.avi)

 > /Volumes/Shared Items/Movies/Animation (Blender3d_SIGGRAPH2005_DVDRip.avi)

 > /Volumes/Shared Items/Movies/Animation (BlenderTricksEpisode.avi)

The datastore is XML. Every import action has it's own Import element. This allows me to do things like undos etc. Oh, and the trick to getting Unicode to work somewhat properly, is to add:

import sys

sys.setdefaultencoding('iso-8859-1')

to /Library/Frameworks/Python.framework//Versions/2.4/lib/python2.4/site-packages/sitecustomize.py

While all of this is very basic ATM, it's already useful for me. No longer do I have to boot up a file server just find out if some music video is stored on it.

Lots of dependencies to work through, but it'll work eventually. Kerberos was something that I didn't bother with (just commented out the relevant lines in the run script) since this is just for a mobile application development project. I also had to comment out line 46 in run.sh.

While reviewing Journler, I stumbled upon a funny command - Toggle Low Light Display. This is the same as System Preferences > Universl Acces > White on Black and is obviously meant for writing in the dark, but it made me wonder if it would also help to conserve battery power. The reasoning being that in a standard black-on-white configuration we're burning pixels that really don't have to be lit at all. Granted, the majority of power consumption in a TFT comes from backlighting, which has nothing to do with the video signal, but I was convinced that there would be a difference. The only question is - how big of a difference?

To set up a consistent environment, I compiled an Automator workflow that triest to simulate a typical writing scenario - grab a few HTML pages, find some words, create a new document in TextEdit, paste the text, pause for 5 mins, check email, repeat (I created the loop by adding an Open Workflow step that opened the same file). The display in this case is a 20" ACD. I also have a bunch of USB devices, but they're not used during the test so they should all be constants. Display Sleep was disabled and brightness was at maximum. The Mac was left to charge overnight between the test runs.

I run the script twice, fiirst with Black-on-White and then the opposite and measure how long the battery lasts, from being totally charged until it completely blacks out.

Luckily, Automator comes with a cli interface making it easy to measure the time. This is not 100% accurate because to stop the timer, I had to reconnect the power cord and kill automator:

> time automator /Users/filipp/Desktop/Loop\ test.workflow/

The results:
Black on white: 54m55.852s
White on black: 52m39.918s

So there's a difference, but it's the exact opposite - WOB actually gave a shorter battery life. I guess this could happen if the Mac is actually doing more graphics processing for the inversion and eating more power as a result. The difference is also so small, that there's really no need for another test.

This came up in the chatroom. How do you protect a folder from being deleted while still being able to write to it? Locking it directly doesn't work, locking the parent one does, but then you can't write to the parent dir. Standard UNIX permissions are of little help, as there is no such thing as a "delete bit". Enter Access Control Lists. First let's make sure they're on:

> sudo fsaclctl -p / -e

and then we can just take away the right to delete:

> chmod +a "filipp deny delete" test

> ls -e

drwxr-xr-x +  5 filipp  filipp   170 Mar 11 19:15 test

 0: user:filipp deny delete

+a always adds an ACE, so to delete one:

> chmod -a# 0 test

Suhweet!

I discovered a nice little shortcut for setting element properties in JavaScroüt. Instead of saying:

nameDiv = document.createElement('div')

nameDiv.className = 'nameDiv';

nameDiv.innerText = showName;

nameDiv.title = showName;

You can "zoom into" the element:

with (nameDiv = document.createElement('div'))

{

  className = 'nameDiv';

  innerText = showName;

  title = showName;

}

A (weird) way to convert PHP arrays to XML. I came up with this for Collective - passing media metadata (with getid3) from the server to JavaScript. It's not supposed to work with numeric indexes - the algorithm doesn't break, it just produces invalid XML (can't have numbers as element names):

function array2xml($array)

{

  foreach ($array as $k => $v)

    {

        if (is_array($v))

          $out .= "<$k " . array2xml($v) . "</$k>\n";

        else {

          $out .= "\n\t$k='$v'";

          if (is_array(next($array)) || (array_pop(array_keys($array)) == $k))

            $out .= ">\n";

        }

    }        

  return $out;

}

A way to convert a file system hierarchy into an XML document, in Python. This is for a simple file-cataloguer I desperately needed (more on this later, hopefully):

imp = minidom.getDOMImplementation()

ldoc = imp.createDocument(None, 'Library', None)



def parsedir(dirname, parent = ldoc.documentElement):

  global ldoc

  if os.path.isdir(dirname):

    newDir = ldoc.createElement('Directory')

    newDir.setAttribute('path', dirname)

    parent.appendChild(newDir)

    parent = newDir

    for i in os.listdir(dirname):

      fp = os.path.join(dirname, i)

      if os.path.isdir(fp):

        parsedir(fp, newDir)

      else:

        newFile = ldoc.createElement('File')

        newFile.setAttribute('name', i)

        parent.appendChild(newFile)

> make love

make: *** No rule to make target `love'.  Stop.

> echo "love" > Makefile; make love

Makefile:1: *** missing separator.  Stop.

Released yesterday. The version number is completely made up. Supports the new IMDb layout and is rewritten to use XMLHttpRequest which should also work through proxies by using the system settings. XMLHttpRequest seems to also give you caching for free (at least it creates cruft in the Cache folder).

This is probably the most succesful thing I've ever made (thanks largely to Martin's nice UI) and this version even got some comments from actual users! 140 downloads in less than 24 hours is not exactly VLC rate, but makes me happy nevertheless. Apple actually featured it again in the Widgets section which always gives you that nice tingly feeling. :)

It's amazing how much energy some feedback gives to a freeware developer. If there's anyone reading this, I would encourage you to send even a small comment or thank-you to your favourite freeware developer(s). It makes all the difference in the world.

A useful little tidbit. To merge the contents of all the files in an arbitrary directory hierachy into one single file, while skipping dot-files:

> find . -type f \! -name ".*" -exec cat {} > /Users/filipp/specs.txt \;

Hit a major bummer while updating the Punk widget today. Essentially, the onreadystatechange handler will miss the redirect/Location: (302) header being sent. IMDb uses 302 when there's only one search result and catching that was a really nice shortcut instead of having to parse the page to find out what has happened. The widget used curl before with which it's super-easy to notice these things, but for some other reasons, I had already re-written everything to use XMLHttpRequest.

Solutions? None so far, at least not for JavaScript. I think I've tried every possible way and place to stick getAllResponseHeaders(), including combining it with setTimeout(), but no cigar. The problem is, this behaviour is in the spec, so there's really no-one to blame. (?)

It really seems that, in my case, the "easiest" thing to do is just taking a peak at the page title. With IMDb it's easy - it's "IMDb Title Search" whenever there's more than 1 result.

I don't know when this happened, but it seems you can't scrape (parse) google.com as easily as before. At least with Python, a simple

s=urllib.urlopen('http://www.google.com/search?q=define%3Aesoteric')

r=s.read()

print r

will give you this notice instead:

/snip/ Your client does not have permission to get URL /search?q=define%3Aesoteric from this server /snip/

And here's me not understanding why my regexes are not matching. :) Apparently, you should at least show some kind of User Agent. In Python you can do this easily by subclassing URLopener and setting it's version property to something, like:

class MyOpener(urllib.URLopener):

      version = "InternetExploiter/666"

urllib._urlopener = MyOpener()

And I'm really starting to love TextMate. Just type your little Python script and hit Cmd-R. Brilliant.