Planet Docunext

Redirecting to Web Application Frameworks

New page

Summary -

Redirecting to Open ID

New page

External Debian Links -

New page: SQL

New page

New page: == Lua SQL Summary == This is fairly straightforward, very logical. I like it! == Lua SQL Example == From YoDNSConf: <pre> --[[ <!-- Program: YoDNSConf Component: domain_list.lua Copy...

New page

New page: Apache Lua module. Category:Lua Category:Apache

New page

Popular Frameworks -

Redirecting to Web Application Frameworks

New page

New page

Email Components -

New page

Programming in Perl -

New page: Oh what fun: <pre> #!/bin/sh cp $1 /tmp/tmp.bk grep '#ZZ\*' $1 > /dev/null if [ "$?" -ne "0" ] then echo "commenting it out" cat $1 | awk ' /#ZZ$/ { printf "%s", $0; next } {print} ' > /...

New page

Example CLI Tasks -

SSI Example -

New page

Official Modules -

My Experience with and Review of Movable Type -

Redirecting to Movable Type

New page

New page: The authentication system for Courier, can use the following mechanisms / back-ends: * MySQL * LDAP * UserDB Category:Courier

New page

New page: This seems interesting... Category:Courier

New page

New page: Courier

New page

Related Pages -

New page: A fast hash database format created by Daniel J. Bernstein. Category:Databases

New page

New page: This is a package for using CDB (constant database) with Postfix. Category:Postfix

New page

Postfix Add-Ons -

External Links -

Time to rethink my ikiwiki setup:

ikiwiki: [NeoCarz Wiki] failed to get lock

No problem though, I’m using Subversion to store the ikiwiki data. For unrelated reasons, I happen to be using NFS to store the subversion repositories.

New page

New page

External Links -

New page: LDAP server powered by Java. Category:LDAP

New page

I’m trying to install encode::detect on debian, but got this error:

cc: error trying to exec 'cc1plus': execvp: No such file or directory

This fixed it:

apt-get install g++

Whoops! After cleaning up some virtual machines, I went too aggressive on some normal machines and removed some necessary stuff. I think I’ll be able to fix this though, as I’m still able to issue commands via ssh.

I reinstalled and re-enabled udev but I was still having problems, so I did this:

ssh example.com "sudo bash -i"
echo "none /dev/pts devpts defaults 0 0" >> /etc/fstab
mount -a

Then I was able to login again.

I’ve updated the iptables script that I use, although it varies from machine to machine, here’s the basic outline:

*filter
:INPUT DROP [0:0]
:FORWARD DROP [0:0]
:OUTPUT ACCEPT [0:0]
-N logdrop
-A logdrop -j DROP
 
-N limitlogdrop
-A limitlogdrop -j DROP
 
-N logok
-A logok -j ACCEPT
 
-N HASHLIMIT
-A HASHLIMIT -m hashlimit --hashlimit 2/hour --hashlimit-burst 1 --hashlimit-mode srcip --hashlimit-name ssh4 --hashlimit-htable-expire=300000 -j logok
-A HASHLIMIT -j limitlogdrop
 
-A INPUT -i eth0 -f -j logdrop
-A INPUT -m state --state INVALID -j logdrop
-A INPUT -s 127.0.0.0/255.0.0.0 ! -i lo -j logdrop
-A INPUT -i lo -j ACCEPT
-A INPUT -p tcp -m tcp --tcp-flags FIN,SYN,RST,PSH,ACK,URG FIN,SYN,RST,PSH,ACK,URG -j logdrop
-A INPUT -p tcp -m tcp --tcp-flags FIN,SYN,RST,PSH,ACK,URG NONE -j logdrop
-A INPUT -p tcp -m tcp --tcp-flags FIN,SYN FIN,SYN -j logdrop
-A INPUT -p tcp -m tcp --tcp-flags SYN,RST SYN,RST -j logdrop
-A INPUT -p tcp -m tcp --tcp-flags FIN,RST FIN,RST -j logdrop
-A INPUT -p tcp -m tcp --tcp-flags FIN,ACK FIN -j logdrop
-A INPUT -p tcp -m tcp --tcp-flags PSH,ACK PSH -j logdrop
-A INPUT -p tcp -m tcp --tcp-flags ACK,URG URG -j logdrop
 
# Rejected ports
# This is used for nolisting, real SMTP servers would have this open.
-A INPUT -p tcp -d 192.0.2.0 --dport 25 -j REJECT --reject-with tcp-reset
 
# Private access ports - trusted sources
-A INPUT -s 192.0.2.1 -p tcp --tcp-flags SYN,RST,ACK SYN --dport 22 -j ACCEPT
 
# Public ports - DNS, Ping, and HTTP
-A INPUT -p tcp --tcp-flags SYN,RST,ACK SYN --dport 22 -j HASHLIMIT
-A INPUT -p tcp -m tcp --tcp-flags SYN,RST,ACK SYN --dport 53 -j ACCEPT
-A INPUT -p udp --dport 53 -j ACCEPT
-A INPUT -p icmp -m limit --limit 100/sec --limit-burst 100 -j ACCEPT
-A INPUT -p tcp -m tcp --tcp-flags SYN,RST,ACK SYN --dport 80 -j ACCEPT
 
# Accept good connections
-A INPUT -m state --state RELATED,ESTABLISHED -j ACCEPT
 
# Drop the rest
-A INPUT -j logdrop
 
# Output filters
-A OUTPUT -o lo -j ACCEPT
-A OUTPUT -p tcp -m state --state NEW,ESTABLISHED -j ACCEPT
-A OUTPUT -p udp -m state --state NEW,ESTABLISHED -j ACCEPT
-A OUTPUT -p icmp -m state --state NEW,RELATED,ESTABLISHED -j ACCEPT
COMMIT

Not all the servers I use support hashtables, so those rules aren’t on all of them. I’m using hastables on kernels that support it because it provides more convenient access, and also foils brute force hack attempts which are just a nuisance.

Since I run debian, I put this into /etc/rc.local:

/sbin/iptables-restore /etc/iptables.sh

Docunext Iptables Page
Docunext Iptables Hashtable Page

Looks like the new version of sitecopy in Squeeze is having problems. I get this error:

sitecopy: Error: Could not connect to server...

I did a quick search and someone else is having the same problem. Doh!

I downgraded to the one from Etch, which requires libneon26, and that works.

Hmmm, I think whatever is wrong here might be due to libneon27, as I’m not able to use svn either!

This might be the problem: Debian Bug report logs - #531338 :: svn: OPTIONS of ‘http://…’: could not connect to server (http://…)

Yup, needed to update libneon27 and libneon27-gnutls from sid.

I’m surprised this isn’t easier to find: Perl S3 Proxy.

Too bad its Apache2 mod_perl2 specific. That’s not really bad, but I would like to use it with NGINX too.

There is also Net::Amazon::S3 and Amazon::S3, maybe those could be run as FastCGI applications?

There are some very nice new NGINX features now available - and the 0.7.x branch has been marked stable!

Here’s the scoop on the new features:

• there is a new image filter which is powered by libgd
• the proxy server now includes a cache

This is terrific news in my opinion. NGINX is really becoming an excellent filtering proxy with the help of the existing XSLT transformer. One thing I’m hoping to see soon is a Markdown filter. That would be sweet.

It took me a moment to figure out how to use the proxy_cache, so here’s and example. In the http configuration section:

http {
    include       /etc/nginx/mime.types;
    default_type  application/octet-stream;
    server_names_hash_bucket_size 128;
    access_log  /var/log/nginx/access.log;
    large_client_header_buffers 4 16k;
    proxy_buffer_size 16k;
    proxy_buffers 8 16k;
    proxy_busy_buffers_size 32k;
    proxy_intercept_errors on;
    proxy_cache_path  /var/lib/nginx/cache  levels=1:2   keys_zone=one:1m;
    proxy_cache_valid  200 302  10m;
}

The proxy_cache_path has to be on the same partition as the proxy_temp_path, and on debian, that is /var/lib/nginx/proxy.

So that’s just the base configuration. To have something get cached, I used the proxy_cache setting in a location directive:

    location /proxyme {
        default_type image/jpeg;
        image_filter_jpeg_quality 70;
        image_filter   resize  100 100;
        proxy_pass http://www.example.com/;
        proxy_cache one;
    }

That’s also an example of how to use the image_filter module. I tried using variables for the resize parameters, but that didn’t work. The resize feature keeps the scale of the original image, which is great.

This was unfortunately much more difficult than I had hoped. What I wanted to do was supply an absolute path for the asset upload, and have the paths for thumbnails built automatically by MovableType.

I was able to hack this into being by specifying the static directives as the asset url and path, like this:

* set target static uri in mt-config.cgi
* override it in the templates:
* include/archetype_editor.tmpl
* include/header.tmpl
* dialog/header.tmpl

First I tried this, but it wasn’t what I wanted:
* Edit /usr/share/perl5/MT/Asset.pm line 83:

: $1 eq 'r'           ? MT->instance->static_path

Also line 62.

ACTUALLY NO - instead changed lines 1004 and 1009 in /usr/share/perl5/MT/CMS/Asset.pm
to %s instead of %r. This will move only the asset, not the thumbnail.

I also used this technique to specify the sub-directory of the static directives.

And MediaUrls helps for general media resources.

I’m using this template to add the javascript required to call out to Google Analytics:

<mt:If name="analytics_code">
<mt:If name="analytics_cookie_path">
<mt:Else>
<MTSetVar name="analytics_cookie_path" value="/blog/">
</mt:If>
<script src="http://www.google-analytics.com/ga.js" type="text/javascript"></script>
<script type="text/javascript">
try {
var pageTracker = _gat._getTracker("<MTGetVar name="analytics_code">");
pageTracker._setCookiePath("<MTGetVar name="analytics_cookie_path">");
<mt:If name="analytics_cookie_domain">
pageTracker._setDomainName("<MTGetVar name="analytics_cookie_domain">");
</mt:If>
pageTracker._trackPageview();
} catch(err) {}</script>
</mt:If>

I setup a cronjob for:

/usr/share/movabletype/tools/run-periodic-tasks

to be run as www-data.

I also symlinked /etc/movabletype-opensource/mt-config.cgi to that directory.

What’s more, I have hacked up a few things which should really be configurable. For example, the template for pop-up windows of image thumbnails is in /usr/share/perl5/MT/Asset/Image.pm (on Debian). I changed it to use thickbox:

 q|%s|,
                 MT::Util::encode_html( $asset->url ),
—
> q|%s|,
>                 MT::Util::encode_html( $popup->url ),
>                 MT::Util::encode_html( $popup->url ),
>                 $asset->image_width,
>                 $asset->image_height,

Then I was trying to get my Multiblog MT install to publish its static resources to a central repository, similarly with uploaded assets. This has proved practically impossible, unfortunately. There is a way I found to do it in a somewhat logical manner… the database stores the asset with a prefix of “%r” which is the blog path. Its possible to use “%s” as the static path, or “%a” for the archive path.

I want better control over the publishing paths for assets and such for caching as well as file distribution. It makes sense from time to time to serve large files like images differently than small html / javascript / css.

I do have to say that the assets functionality it pretty awesome. They’ve done some really impressive work on managing images - great work Six Apart!

I love workarounds. This problem about javascript same domain rules conflicting with MT’s static web path configuration option had me stumped.

Then I remembered mod_rewrite!

RewriteEngine On
RewriteCond %{QUERY_STRING} template
RewriteRule . - [E=MT_CONFIG:/usr/lib/cgi-bin/movabletype/mt-config-admin.cgi]

I just setup a new Debian OpenVZ container on a commercial VPS. It was installed as Lenny, but since I want to install a newer version of Varnish, I upgraded to Squeeze.

uh-oh::/var/cache/apt/archives# dpkg -i dpkg_1.14.26_i386.deb
(Reading database ... 12216 files and directories currently installed.)
Preparing to replace dpkg 1.14.26 (using dpkg_1.14.26_i386.deb) ...
Unpacking replacement dpkg ...
Replaced by files in installed package base-files ...
Setting up dpkg (1.14.26) ...
touch: setting times of `/var/log/dpkg.log': Bad address

Oh-no! A little digging reveals that there is a conflict between the newer version of coreutils and the kernel that is running on the openvz host. I’m surprised as I have an openvz host which is running 2.6.18 with squeeze containers… I think. I’ll have to check that.

I’ve been hard at work lately - just finished up some cool tasks though, including some work with a Perl MVC framework called Catalyst. I like it! I’ve also been using Nginx’s pre-compression gzip module to lower server load.

I’m also using Movable Type and really growing to like publishing with static files. During the process of embracing static files, I’ve come to learn more about caching, cookies, and HTTP headers. In short: cookies crash the caching party. Well its not that serious, but in the never ending pursuit of performance, every little bit counts!

I finally got around to updating my Varnish proxies to version 2.0. Nice work guys! Very very nice. I’ve got the health checks and fail-overs operational. Sweet!

Lastly, I was able to round up the code I used to power the direct “putting” of a text document to a DAV server, which in turn is connected to a Subversion repository with auto-commit enabled. My goal is to connect this with an Ikiwiki backend which will get triggered by the post-commit-hook and the publish the new content. My plan is to document and package this recipe up nice at a shiny new domain: http://www.jaderz.com/blog/. (Its empty right now, you don’t have to visit.)

That’s all for now… thanks for listening.

I just had a nice conversation which included the topic of Ming: a C++ library which can generate SWF movies. It has bindings for many scripting languages like PHP, but can also by access with compiled C code. I’ve been using it for years and recently I’ve taken a little break, but I really want to start using it again.

Here is the Docunext wiki page about Ming, and here’s an open source project I’ve been working on for quite some time which uses Ming:

Internet Floor Plans

And of course, what’s a blog post about Ming without a link to Gazb’s Ming tutorials?

$blog_id = 0;

my $system_perms = $user->permissions(0);

my $system_perms = $user->permissions(0) unless $perms && $perms->blog_id;

I’ve got a few hours to work on something good… should it be WebDAV + SVN integration with Ikiwiki, or upgrading my Varnish installs?

Truthfully, even though I want to upgrade ASAP, I think the Varnish installs should wait a few more weeks. I’ve been tweaking some settings and want to see what effect they have before I go changing anything.

So WebDAV + SVN + Ikiwiki integration it is. I’m planning on hooking up Established Sites to this somehow too; maybe even tying in MovableType!

I was super excited about this when experimental support was first implemented, and now that its been mainlined for a few days I’m a little surprised at myself that I haven’t gotten it going yet.

I’m doing it now, and I’m hoping it works as well as I think it will! I’m surprised though… there hasn’t been much activity on the Epiphany subversion server lately. Hopefully I’m using the right address. Midori is staying active, speaking of which I wonder if the Webkit GTK+ gnome keyring integration works with Midori as well?

I just recompiled Midori and saving passwords for http auth isn’t available, but I’m compiling it again after doing a clean, maybe that will help.

I’m really looking forward to when I get to use Epiphany webkit all the time. The older version I’m using (2.22.3) is super fast, but is missing some important features:

• Gnome Keyring (actual configured via Webkit)
• Selective blocking of popup windows

I just tried compiling the latest subversion revision; it compiles, but when I try to run it I get this error:

symbol lookup error: /usr/local/bin/epiphany: undefined symbol: gtk_entry_set_icon_drag_source

Doh!

UPDATE! I got it to work, the gtk libs in /usr/local/lib were old, so I copied them from /usr/lib. That move crashed my current X session so after a reboo, voila:

Also got the Gnome Keyring inclusion to configure, and it appears to work, but only for HTTP Auth.

Anyone else getting this? It happened to me once before when I was trying some gtk libs from sid, so I back-graded to lenny and it worked again.

It happens with all print dialog boxes, but with Iceweasel its instantaneous. I had previously tried to diagnose this but strace revealed little and I didn’t know what else to try…

Hmm, looks like Iceweasel is randomly crashing too. I just recently upgraded it, so there must be something funky going on. Seems to be related to my openvz kernel??? (The random crashes, not the print dialog crashes.)

UPDATE: For some unknown reason - the crash is no longer happening.

Varnish 2.x is now is Debian Squeeze, and it includes some cool failover functionality I want to give a whirl. The 2.x branch has actually been around for awhile, but since Varnish is the frontline gateway to most of the web sites and services I publish, I wanted to give it some time to get tested.

I’ve been testing it too, and everything seems fine to me. I checked if there was a backport of Varnish 2.x for lenny, but I coulnd’t find one. That would be great if it were possible!

Anyway, I’m dreading having to update the syntax of the vcl files. I don’t think that they are too different, but my vcl files are messy and disorganized. In preparation of the upgrade, I’ve at least started to clean them up a bit.

UPDATE: I just tried my version 1 vcl with version 2 and the switch was really easy, I just used this in vim:

:%s/set backend//g

Preparing to replace libxml-sax-perl 0.96+dfsg-1 (using libxml-sax-perl_0.96+dfsg-1_all.deb) ...
Unpacking replacement libxml-sax-perl ...
Setting up libxml-sax-perl (0.96+dfsg-1) ...
update-perl-sax-parsers: Registering Perl SAX parser XML::SAX::PurePerl with priority 10...
Fatal: cannot call XML::SAX->save_parsers_debian().
You probably have a locally installed XML::SAX module.
See /usr/share/doc/libxml-sax-perl/README.Debian.gz .
dpkg: error processing libxml-sax-perl (--install):
 subprocess post-installation script returned error exit status 1
Processing triggers for man-db ...
Errors were encountered while processing:
 libxml-sax-perl

I was able to work around this using:

cd /usr/local/share/perl/5.10.0/XML/
mkdir dnu
mv SAX* dnu/
apt-get install libxml-libxml-perl

./.libs/libwebkit-1.0.so: undefined reference to `WebCore::XMLTokenizerScope::XMLTokenizerScope(WebCore::DocLoader*, void (*)(void*, char const*, ...), void (*)(void*, _xmlError*), void*)'
./.libs/libwebkit-1.0.so: undefined reference to `WebCore::XMLTokenizerScope::currentDocLoader'
./.libs/libwebkit-1.0.so: undefined reference to `WebCore::XMLTokenizerScope::~XMLTokenizerScope()'
./.libs/libwebkit-1.0.so: undefined reference to `WebCore::XMLTokenizerScope::XMLTokenizerScope(WebCore::DocLoader*)'
collect2: ld returned 1 exit status
make[1]: *** [Programs/DumpRenderTree] Error 1
make[1]: Leaving directory `/home/albertlash/src/webkitsvn'
make: *** [all] Error 2

I’m not complaining, just noting.

# emerge -uDpv world

These are the packages that would be merged, in order:

Calculating dependencies... done!
[ebuild     U ] sys-libs/timezone-data-2009f [2009d] USE="nls" 360 kB
[ebuild     U ] sys-apps/sandbox-1.9 [1.6] 320 kB
[ebuild     U ] sys-apps/less-429 [424] USE="unicode" 0 kB
[ebuild     U ] sys-apps/man-pages-3.20 [3.19] USE="nls" LINGUAS="-cs -da -de -es -fr -it -ja -nl -pl -ro -ru -zh_CN" 1,062 kB
[ebuild     U ] sys-fs/udev-141 [140] USE="(-selinux)" 438 kB
[ebuild     U ] sys-devel/autoconf-2.63-r1 [2.63] USE="-emacs" 0 kB
[ebuild  NS   ] sys-libs/db-4.7.25_p3 [4.3.29-r2, 4.5.20_p2-r1, 4.6.21_p4] USE="-doc -java -nocxx -tcl -test" 12,829 kB
[ebuild     U ] dev-libs/libpcre-7.9-r1 [7.8] USE="bzip2 cxx (unicode) zlib -doc" 842 kB
[ebuild     U ] www-servers/nginx-0.7.50 [0.7.43] USE="pcre perl ssl zlib -addition -debug -fastcgi -flv -imap -random-index -status -sub -webdav" 567 kB
[ebuild     U ] sys-apps/coreutils-7.2 [7.1] USE="acl nls -caps (-selinux) -static -vanilla -xattr" 0 kB
[ebuild     U ] sys-apps/grep-2.5.4-r1 [2.5.4] USE="nls pcre (-static%)" 0 kB
[ebuild     U ] app-admin/eselect-1.0.12 [1.0.11-r1] USE="-bash-completion -doc (-vim-syntax%)" 143 kB
[ebuild  N    ] app-admin/eselect-python-20080925  2 kB
[ebuild     U ] www-servers/varnish-2.0.4 [2.0.3] 769 kB
[ebuild     U ] dev-lang/ruby-1.8.7_p72-r11 [1.8.7_p72-r10] USE="berkdb gdbm ipv6 ssl -debug -doc -emacs -examples -rubytests -socks5 -threads -tk -xemacs" 0 kB
[ebuild  NS   ] dev-lang/python-2.6.2 [2.4.4-r13, 2.5.4-r2] USE="berkdb gdbm ipv6 ncurses readline ssl threads xml -build -doc -examples -sqlite -tk -ucs2 -wininst" 10,905 kB
[ebuild     U ] dev-python/pycrypto-2.0.1-r8 [2.0.1-r6] USE="-bindist -gmp -test" 151 kB
[ebuild     U ] sys-apps/openrc-0.4.3-r2 [0.4.3-r1] USE="ncurses pam unicode -debug" 0 kB
[ebuild     U ] sys-apps/busybox-1.13.4 [1.13.3] USE="pam -debug -make-symlinks -savedconfig (-selinux) -static" 2,000 kB
[ebuild     U ] sys-apps/shadow-4.1.3.1 [4.1.2.2] USE="cracklib nls pam -audit (-selinux) -skey" 1,661 kB

Total: 20 packages (17 upgrades, 1 new, 2 in new slots), Size of downloads: 32,043 kB

I finally found the epiphany subversion repository so I’m trying to build it.

Requested 'gtk+-2.0 >= 2.16.0' but version of GTK+ is 2.14.7
Requested 'gtk+-unix-print-2.0 >= 2.16.0' but version of GTK+ is 2.14.7

I installed the libgtk from sid which allowed it to compile and install, but I get this error when I try to run it:

undefined symbol: gtk_entry_set_icon_drag_source

Hmm, can’t figure that one out. Guess I’ll have to wait for some upgrades.

Ugh, several hours later I realized that by upgrading these packages:

ii  gtk2-engines-pixbuf                    2.16.1-2                           Pixbuf-based theme for GTK+ 2.x
ii  libgail-common                         2.16.1-2                           GNOME Accessibility Implementation Library -- commo
ii  libgail-dev                            2.16.1-2                           GNOME Accessibility Implementation Library -- devel
ii  libgail18                              2.16.1-2                           GNOME Accessibility Implementation Library -- share
ii  libgtk2.0-0                            2.16.1-2                           The GTK+ graphical user interface library
ii  libgtk2.0-dev                          2.16.1-2                           Development files for the GTK+ library

I could no longer print in iceweasel. It would cause a segmentation fault and crash / quit immediately after I tried bringing up the print dialog. I downgraded with something along these lines:

apt-get install libgtk2.0-0=2.14.7-5 libgail18=2.14.7-5 libgtk2.0-dev=2.14.7-5  gtk2-engines-pixbuf=2.14.7-5 libgail-common=2.14.7-5  libgail-common=2.14.7-5  libgail-common=2.14.7-5 

and now I can print again. I found a bug in the debian database about a potential conflict with libcairo. Who knows?

As a follow-up to this post from two years ago (!!) How I use Apache behind pfSense, I think I may have solved a similar problem with HTTP authentication, trac, and the pfSense sticky states balancer.

I wrote up a little javascript keepalive (will publish soon), turned off the Keepalive in Apache, and it seems to work fine. Previously it would log me out randomly, because the load balancer had switched backends.

Hmmm, its very strange because it just switched again. I wish I knew why it was not remaining a sticky connection. The blurb doc says as long as there is a connection, it stays sticky. Hence the keepalive, but its apparently not working.

Aha - I think I may have figured it out. I was connecting to other back-ends through the load balancer which had different keepalive settings. I wish the keepalive settings worked.

There are many facets to this situation: http keepalive, tcp keepalive, pf states, proxies, and more I’m not thinking of at the moment. Its very interesting though.

There are many types of user agents, but I’d rather not have to keep track of them all. I’ve been thinking about trying to keep it simple, so I’ve come up with these categories as an idea:

• msie - I do not like Microsoft products
• standard - I like standard compliant browsers
• bots - web crawlers, w3m, lynx, etc.

This is wonderful news: the Debian GNU/kFreeBSD distribution is now official. I’ve been very impressed with this port for quite some time now and am very pleased to see it available in all the debian repositories.

Congratulations to all the debian people on another huge step forward!

If you want to help out, there is a TODO file for the Debian GNU/kFreeBSD port here.

Markdown is a very simple syntax for stylizing documents. I really like it because it works for both plain text ASCII documents and when its converted to HTML or something similar.

I’m starting to use it for letter writing, something I enjoy doing very much. I write the letter using Markdown syntax and then convert the text file to PDF using, what else, markdown2pdf, part of pandoc. My first few letters were hardly formatted at all just using the resulting HTML layout features: paragraphs and line breaks.

Reading up on the pandoc tools, I decided I needed to learn about latex documents. That was the key - I was able to get a larger font size with this latex header:

\documentclass[12pt]{amsart}
\setlength{\parindent}{0pt}
\setlength{\parskip}{12pt plus 2pt minus 1pt}

and this command:

markdown2pdf -C test.latex my_markdown_letter.txt

Recent Webkit builds were crashing on my Wordpress sites and I finally tracked down the problem. The K2 theme sends a Pragma header for some javascript files, and removing that header call stopped Webkit from crashing. Hooray!

Webkit shouldn’t crash because of that header, but I don’t want K2 to send it anyway.

UPDATE: Actually it was the charset text after the content-type header:

-	header('Content-Type: text/javascript; charset: UTF-8');
+	header('Content-Type: text/javascript;');

A recent run of powertop discovered that the webcam in my Acer Extensa 5620Z was causing a significant amount of wakeups. I never use it, in fact its covered up with a sticker.

[    6.888500] usb 1-1: Product: Acer CrystalEye webcam
[   15.287631] uvcvideo: Found UVC 1.00 device Acer CrystalEye webcam (064e:a101)
[   15.288506] input: Acer CrystalEye webcam as /class/input/input6

zzillezz.net turned a similar webcam off with “blacklist gspca”. I’m using “blacklist uvcvideo”.

While I was at it, I also blacklisted a few other modules to save RAM as my laptop recently lost 2GB of RAM.

I’m surprised to learn that graphics powerhouse Silicon Graphics is being sold for only $25 million dollars to Rackable Systems. I hadn’t heard their name for a few years, but I always regarded them as a quality company.

I read a little into the story and learned that they have a few hundred million dollars worth of assets, but several hundred dollars worth of debt. Ouch!

I was reading up on Webkit optimizations and one of the suggestions was to use hardware acceleration on the video card. I’ve experimented extensively with graphics acceleration on my Everex Via C7 notebooks, but I’ve refrained from toying with the xorg.conf on my Acer Extensa 5620Z, simple because it worked out of the box with debian and I had no complaints.

I figured I’d give is a shot, found an xorg.conf configuration which said it was for the Intel Corporation Mobile GM965/GL960 Integrated Graphics Controller, and lo and behold, a strange bug which had been plaguing my Iceweasel for months went away. I’m not sure if the two are related, but I’m *so* glad that the bug is gone.

I would have tried to track down the source of the problem before, but the problem is tough to describe. It only affected background images, and only affected them sometimes. The would appear 50% vertically offset - or so it would seem, something funky was going on with them.

I’m hoping that the new configuration actually enables some hardware acceleration, and maybe even cause my laptop to run a little cooler? Is that too much to ask for?

I just found the solution to an annoying bug:

“[Exception… “‘Syntax error, unrecognized expression: [@rel*=lightbox]’ when calling method: [nsIDOMEventListener::handleEvent]” nsresult: “0×8057001e (NS_ERROR_XPC_JS_THREW_STRING)” location: “” data: no]”

This started after I upgraded jQuery to 1.3.2 from 1.2.x. The fix is:

Index: jquery-wp-lightbox/js/jquery.lightbox.js
===================================================================
--- jquery-wp-lightbox/js/jquery.lightbox.js	(revision 654)
+++ jquery-wp-lightbox/js/jquery.lightbox.js	(working copy)
@@ -501,7 +501,7 @@
 			var groups_n = 0;
 			var orig_rel = this.rel;
 			// Create the groups
-			$.each($('[@rel*='+orig_rel+']'), function(index, obj){
+			$.each($('[rel*='+orig_rel+']'), function(index, obj){
 				// Get the group
 				var rel = $(obj).attr('rel');
 				// Are we really a group

I also want to note that I had to do some serious snooping with K2 to figure out why jQuery’s dollar sign was causing a problem. Turns out that the k2 functions script disables it with:

jQuery.noConflict();

After commenting that out, it works!

Tidy is a nice little library which is used to to clean up HTML. Its been around for awhile, and from what I understand, it has been retained and maintained by a group of users after the original author, Dave Raggett, decided to halt upkeep.

There are many language bindings, and I’ve recently been using a lot more for converting HTML into valid XML, which can be transformed by XSLT into XHTML. Good stuff!

Docunext Wiki Page on Tidy

I’m experimenting with PHP, CSS, and XML, trying to come up with some sort of css2xml script, while considering the idea of mangling classes and identifiers to prevent collisions when creating mashups.

<?php
require_once 'HTML/CSS.php';
require_once 'XML/Serializer.php';
$mycss = file_get_contents('test.css');
 
//$mycss = preg_replace('/#/','id___',$mycss);
//$mycss = preg_replace('/\n\./','class___',$mycss);
 
$css = new HTML_CSS();
$css->parseString($mycss);
 
//$arr = $css->toArray();
//var_export($arr);
ob_start();
$css->display();
$converted = ob_get_contents();
ob_end_clean();
//echo $converted;
 
//exit;
$mycss = preg_replace('/\n#/','id___',$converted);
$mycss = preg_replace('/\n\./','class___',$mycss);
$css->parseString($mycss);
$arr = $css->toArray();
 
//echo $converted;
//echo $mycss;
//exit;
 
$options = array(
  XML_SERIALIZER_OPTION_INDENT        => '    ',
  XML_SERIALIZER_OPTION_RETURN_RESULT => true
);
$serializer = &new XML_Serializer($options);
 
$result = $serializer->serialize($arr);
echo $result;
?>

Note, this code is just a sketch, and I don’t expect it to function well.

Also, of the post I made recently about css parsers, this one in ruby again caught my attention.

Lastly, I found csstoxml. Its written in java, so I’m a little hesitant to try it out due to my lack of experience with java.

I kept on getting:

/usr/bin/gem:14: undefined method `ruby_version' ...

on a debian machine whenever I tried using gem. I ended up commenting out a few lines, then installing the latest version of rubygems-update from here with:

gem install rubygems-update-1.3.1.gem

Looks like it worked, but not for the real problem I was trying to solve. This was to try and solve this problem:

ActionView::TemplateError (undefined method `[]'

Which was solved thanks to these pages:
Re: [SOVED] AJAX error: ActionView::TemplateError (undefined method `[]’..
and
undefined method for enumerable

The actual fix was to put:

unless '1.9'.respond_to?(:force_encoding)
  String.class_eval do
    begin
      remove_method :chars
    rescue NameError
      # OK
    end
  end
end

at the beginning of config/environment.rb.

Just some notes.

Requirements

* Libsoup
* Libproxy

apt-get install build-essential libcurl4-gnutls-dev  libgstreamer0.10-dev libgstreamer-plugins-base0.10-dev
apt-get install libgconf2-dev
apt-get build-dep webkit libsoup
svn co http://svn.gnome.org/svn/libsoup/trunk libsoup
svn checkout http://libproxy.googlecode.com/svn/trunk/ libproxy-read-only
svn checkout http://svn.webkit.org/repository/webkit/trunk WebKit
CFLAGS="-msse -msse2 -msse3 -mtune=nocona" CXXFLAGS="${CFLAGS}" ./configure
make -j4

UPDATE - I’m trying out x86_64 support for JIT Javascript. (See 32 or 64bit for my complaints about architecture limitations.)

Result - I was able to compile Webkit and JIT javascript optimizations and it runs with epiphany, but I’m not sure if its really doing anything.

I have to say that it is indeed tempting from time to time to switch back to 32-bit. Occasionally I find programs which run only on 32-bit: flash, squirrelfish, and chromium.

But I’m sticking with 64 bit as much as I can. I will obviously continue to use 32 bit on machines that don’t support 64! That includes some of my development machines, a backup notebook, a netbook, and my pfSense firewalls.

After briefly thinking about Javascript parsers, I did a quick search for CSS parsers, and found, as El Guapo might say, a plethora of piñatas.

Here’s a few which caught my attention:

• http://cssparser.sourceforge.net/
• http://csstidy.sourceforge.net/
• http://ajax.sourceforge.net/cypress/
• http://code.google.com/p/ruby-css-parser/
• http://code.google.com/p/cssutils/

I’m really intrigued that there are so many parsers written in Ruby.

I’ve been evaluating different strategies for caching parts of an XHTML page, like a header, footer, or sidebar. I’ve come to the conclusion that the best way to do this is to store the xhtml as a dom object in javascript. It could also be stored as plain XHTML, but then there are issues with multi-line strings in javascript.

It would be possible to use E4X (EMCAscript for XML), but that is not widely supported by browsers yet.

So anyway back to Javascript DOM code. Its not as simple to write as HTML, and there are many convenience packages to help this, but I’m under the impression that may reduce its efficiency.

I did some searching on ways to generate Javascript DOM code from XHTML found html2dom. It does exactly what I was looking for, but the code doesn’t have an explicit license. The author states on another page that the code is in the public domain, but I would feel more comfortable if that were included in the code itself.

Then there is the issue that I don’t have a server-side javascript interpreter setup. These are somewhat rare, though they are getting much more prevalent, and I expect that to continue. Since I’m a big fan of XSLT, I’ve decided to try writing a stylesheet which would convert xhtml to dom code. Its very rough at this point, but I’m checking it into the PBooks repository and will post a link once its published. The only real hurdle I ran into was dealing with multi-line text nodes. As usual, they cause some problems with javascript, but I just used the translate function to remove all newlines. I also ran into an issue that the style attribute is read-only. Not a big deal, I need to refrain from using in-line CSS anyway. The stylesheet is basically an identity transform that converts XML to Javascript DOM instructions. Another cool thing about this is that the resulting dom code can be gzipped and cached as long as it doesn’t change too much. That is a good thing!

Here is a link to the first revision of xhtml2dom.xsl.

I like how this is working. For some reason, I’m now thinking about javascript parsers. Does such a thing exist? I think it would be possible to create one using Ragel. Here is one implemented in Ruby: rbNarcissus (and a backup of the code here).

UPDATE: Instead of the translate function for newlines, I’m using the normalize-whitespace function suggested here at xmlplease.com/whitespace.

The pfSense hackathon is over, and I’m skimming through the change logs. Looks like most of the changes were to the 2.0 branch, which I’ve only tried on a virtual machine, but it looked nice.

Clog is gone, replaced by fifolog, and is it true? Ftp proxy is gone? Nice, I never liked that. Instead I use NAT for a few hundred ports in the 65000 range, and that seems to work much better. I set the same range in proftpd.conf and its good to go. Looks like they are using the prototype javascript framework ( I prefer jQuery ).

In other news, the pfSense folks are adding some services to the pfSense offering, and so far it looks terrific. There is now (or soon to be) the ability to save your configuration to a secure service. Nice work people!

I should also note that the glxsb (Geode LX Security Block) kernel module continues to run well on my ALIX board. Excellent!

I was surprised to find that Ubuntu has a package for Chromium - the open source browser from Google. It uses the WebKit HTML engine, and a new javascript engine called V8.

I love trying out new stuff, so I managed to hack it into my debian laptop, but (I guess due to ia32 / gtk differences) the browser won’t display any pages, everything just shows up blank.

I then started to try and install on a 32-bit ubuntu machine, but of course I also decided to upgrade it to Jaunty Jackalope. Jaunty is still in alpha, so I wasn’t surprised to find its wasn’t a totally smooth upgrade. I’m still working on it… well the upgrade is done and everything looks good. Unfortunately, I’m still getting blank screens with Chromium.

Turns out I just needed to install the Microsoft fonts installer package. After that, it started to display pages. Let’s see if that works for on my Debian 64-bit machine:

apt-get install ttf-mscorefonts-installer

Nope, I guess I already had that installed.

I noticed on the opening screen that blogging is discouraged. That’s too bad, I disagree with the statement that it doesn’t help.

I’m sticking with XSL, XML and DOM, so I’m trying to free myself of strings. I had a tough time dropping disable-output-escaping, as I first tried xsl:copy-of, which resulted in an empty xmlns attribute in xhtml docs.

I finally remembered that the nodes just have to be rebuilt, either using xsl:element or the new elements entirely. Now to try and keep things flexible, I’ll need to craft up some xsl:templates for common html elements. So far I’ve got script and link, but of course I’ll need a whole bunch more than that.

Also, attribute sets!

Hah, well its actually not that difficult. All that was needed was an identity transform which avoids copy and copy-of. Thanks to Ben, its really quite simple!

<xsl:template match="node()">
<xsl:element name="{name()}">
<xsl:apply-templates select="@*|node()"/>
</xsl:element>
</xsl:template>
 
<xsl:template match="@*|text()|comment()|processing-instruction()">
<xsl:copy/>
</xsl:template>