Log of unix software lessons, including a Zareason lightlap laptop (based on an Asus S62E) running Ubuntu Gutsy, Hardy, Intrepid, Jaunty, Karmic, ...

Christopher Barrington-Leigh

See also a blog, to which I may be transitioning, at http://cpbl.wordpress.com.

Older notes, including a Toshiba Satellite 2410 laptop and a server, are available separately linuxInstallationLessons-toshiba.html

In 2003 I left the world of professionally supported Unix and got my own laptop. I'm now on my second. These are just my notes on things I've learned as a user of less-than-fully-supported (open source) unix distributions from Ubuntu. Some things are written as advice to others; some as notes to myself. I include some notes on recently developed software I like, too.

[2006] For those who are just looking for a good, working operating system but are frustrated with Microsoft, I endorse the following advice (from someone more knowledgeable, 2005):

If you're buying something new, buy a Macintosh, supplemented with Openoffice.org's free office suite (or its kin). When you can now get (in Mac OSX) a unix core underlying a polished GUI, why bother with GNU/Linux? If you still want to, make sure you have a reason.
If you are already invested in intel-like hardware or, like me, wedded to the unix environment, Ubuntu does well. If you're running Ubuntu, you can find beautiful, clear answers and methods without strange gui interfaces at: http://ubuntuguide.org/.

My current status:

[2008] I sold my Toshiba Satellite 2410 linuxInstallationLessons-toshiba.htmlafter five years of great service. I have bought a Zareason Lightlap, fully loaded with GNU/Linux.

1. General wisdom

1.1 Buying a computer

Vote with your dollars. Price satisfaction is usually pretty relative, so don't be fooled into needing to buy the cheapest deal you can find. But do be strict about buying from someone who is NOT paying Microsoft for an OS that you won't use.

Dec 2007: Here is one good site listing companies selling laptops/computers without paying a Microsoft tax: http://mcelrath.org/laptops.html

They really do exist now (tablet pcs too), likely in your country, but expect that their prices reflect lower sales volumes.

Image zareason_small Jan 2008: I have chosen to buy from http://zareason.com (in Berkeley, two blocks from my old house). They have great community- and ecologically- minded origins and practice, low prices, will ship worldwide (I am in Canada and there is not yet any local firm that comes close in quality) and to a different address than your billing address, and have a constructive, open attitude to support and service.

After consulting with them, I got a fully set-up machine fine-tuned for its GNU/Linux operating system, ostensibly with no uncertainties about hardware support, no installation necessary, and even, at my request and for no charge, some customised software installed. Truly, though UNIX is dead, the awkward period in the UNIX $\rightarrow$ GNU/Linux transition years of self-support are coming to an end... gradually. The hassle of having to install your own operating system or search by yourself for compatible hardware, or fend for yourself when something goes wrong, or compromise by buying Windows only to throw it away — these things are no longer necessary. My computer comes without an implicit Microsoft tax, without any MS Windows sticker or MS Windows key (instead, Ubuntu sticker and key) and, as shipped, with possibly glitzier interface and better functionality than either Windows or Macintosh offer.

1.2 Rules for installing GNU/Linux on a “migrating” friend's computer

[2005] Having messed this up once, I suggest the following (to myself, at least) if someone wants help making a Windows / GNU-Linux dual boot system on one hard drive:

Never use Partition Magic after GNU/Linux is installed.
Never get involved in helping with Windows issues. Never get involved in backing up someone's data. Give the person Partition Magic, and let them back up all their data (e.g. by network) and shrink the size of their NTFS partition to contain just the OS (and they, using Windows, or you during the install can create a new FAT32 data partition to be shared by both OS's) [Update: 2007: No need for FAT32! GNU now has safe read/write ability for NTFS, including Windows Vista!]. You can point out the advantage of separating Windows OS from Windows data, regardless of installing a new OS. You can point out the advantage of backing up one's data because... Duh.. data should be backed up.
Then come along and installing GNU/Linux will be a breeze.

1.3 GNU/Linux partition choice

The advice above about Partition Magin may be obselete given the modern open source partitioning tools. They are reliable and powerful.

In any case, many distributions still make a terrible default choice for partitioning. If you have space, you should make the following partitions, ie using the manual or advanced options during the partitioning step (May 2010). Where appropriate, choose the most up to date Linux partition type, e.g. ext4.

First OS: “/”, probably 7-15 GB
Second OS: “/altOS”, or etc, probably 7-15 GB
Data: “/home”: the bulk of your disk space. I tend to put all non-open-source software on /home/local, so that I don't need to reinstall that, either
swap (once or twice the size of my RAM)
If you want some other buffer space, or have a Microsoft OS, or a FAT or NTFS drive, leave that here too, e.g. “/windoze”
You can make others. e.g. if it's a server, you may want to make /var and /tmp their own partitions. I prefer not to.

This way, if updating GNU/Linux distributions, you can alternate between the two OS partitions. And you need never backup your /home directory just to install a new OS.

2. Zareason Lightlap review: laptop purchased January 2008

Despite my enthusiasm for buying a turnkey linux distribution, I have not been satisfied with my first laptop from Zareason. They have a reputation for excellent support and followup which you will not get with big sellers, however this was a bit erratic in my first month due to the company being rather busy rolling out a new product. In February 2008, they apologised for this and for the difficulties I was having and offered for me to return the laptop for an upgrade. That would, however, be a hassle.

Late in 2008 my DVD drive stopped working. Zareason shipped out a replacement. However, after having used the drive probably only a couple of times over the few months since installing the new one, the new one failed the same way. What a hassle.

2.0.1 Hardware review:

It's an Asus S62E, with any mods Zareason has done.

The down-arrow key is noisy. This appears to be a design flaw rather than a construction problem particular to my machine. No other keys are noisy.
There is no external (hardware) volume dial like on my Toshiba. I miss that.
When I received the computer, it rattled. Zareason computers come with a screwdriver (with a ribbon and bow) to emphasize their open hardware support standard. However, this screwdriver turned out to be entirely symbolic, since it was the wrong size for my laptop. I opened some panels on the bottom and found that indeed, a completely loose screw was rattling around knocking all the interior components. It had come out of the CPU heat sink and in fact another of the heat sinks screws (though lock-tighted) was also on its way out.
One of the panels on the bottom is squishy. That is, its padding pushes the panel out a bit, so it is not lying flat/flush, but is bowed outwards a bit and is squishy.
I ordered the larger, 9-cell battery, but there was no indication on the order form (web site) that this battery does not fit within the form of the laptop. That is, the battery sticks out two centimetres from the back of the laptop! I like the long battery life, though.
The touchpad on the laptop came poorly set up: the vertical scroll function covers more than a quarter of the pad, rather than just the right hand edge. Zareason has not found a solution for me after over a month. [Eventually, they have! April 2008: scrolling is now just on the right edge where it should be]
I would have preferred a more capable video card than the Intel X3100. I was led to believe it was sufficient for modern desktop effects, but is in fact insufficient for the needs I made clear to Zareason before purchase. The machine I got, their “Lightlap,” sold with the description “Compiz works fine” with a small tweak which will be fixed in the next OS update. It turns out the graphics chip is blacklisted for compiz and only runs with the best-performance video playing turned off, and the next release of Ubuntu won't have it fixed. I had told Zareason I needed to use it in extended desktop mode with an external monitor, and wanted to be able to switch to an external projector on the fly for presentations, but there was additional set up (some of it still not provided) required to do these things and in fact the graphics card is incapable of running Compiz with a working desktop when an extended desktop is side by side with a monitor of reasonable resolution. Normal, most basic, compiz functions become unbearably slow when more than a few windows are open. This is a major disappointment for me with this purchase, and is actually not consistent with what Zareason promised me.
Sleep and hibernate mode were promised to be “rock solid”. I haven't tried hibernating much, since with 4 GB of RAM it takes forever (?), but there is a major bug with sleeping. Under both the Gutsy that Zarason installed and the latest Hardy, anytime I try shutdown after having previously used sleep mode at least once during the session, my laptop hangs when shutting down. It stops right when the progress bar is finished and it looks like it's all done. I have to power cycle it to reboot. The next boot, the BIOS clock date has been reset to Jan 1 2002 (?), while the time remains correct. If I continue booting with this date, it naturally makes various things incredibly upset, and X login fails. I fix the clock using a tty login but the subsequent time I boot I have to wait for 25 minutes while all my drives get fsck'd, since they appear to the system not to have been checked since 2002. (A better way to boot after having used sleep is to enter BIOS and reset the date there before booting up). Five months after buying the computer, sleep mode is still unusable. By version 8.10 of Ubuntu, the sleep mode worked perfectly.
The browser-launch and one other shortcut key are not recognized by the OS.
2009: The CD drive has come loose internally, twice, somehow, the symptom of which is that it stops being able to mount disks.
2009: A short developed in the cable controlling the display, so the screen is not usable when the machine is turned on unless the lid is nearly closed. This is just a bit after the 1 year warrantee is up. But I suspect that it is caused by my having dropped the thing once and breaking a hinge inside.
2010 May: One of the two 2GB memory chips died. This does not usually happen so soon with well-matched hardware. Machine works fine once the bad chip was identified and removed. By the way, the sleep and video problems above are completely gone or much alleviated with recent upgrades.

2.0.2 Software

The laptop comes without software to use the built-in “webcam”. It works fine with ekiga for video conferencing (out of the box), but there was no software for viewing and taking images with it. Two options in the repositories, “cheese” and “camorama”, crash or fail on the machine!
I had efax installed, but it does not work with the built in modem. Other vendors have got this modem to work, but Zareason has not yet figured out how to set it up, and that does not seem to be a high priority for them.
Due to the limitations with compiz, I ended up installing Hardy alongside Gutsy after not too long, somewhat defeating the whole purpose of buying a premium new machine from a Linux laptop supplier.

Zareason installed some custom software for me, but of course could not do so for some sketchy-legality software. Visit the following site for instructions on getting the last things:

https://help.ubuntu.com/community/Medibuntu

2.0.3 64 bit is still not standardly supported

This is surprising and frustrating to find. Proprietary software like Skype, etc do not support 64 bit yet!? There is a whole game of installing 32-bit libraries to make things work (for instance, my license for Stata is apparently only for 32 bit machines!!); I am trying to avoid that.

2.1 Microphones

I do not understand the microphones. There is a built in one and there is a jack for a mic. The gnome ALSA level controls refer to “front” and “front mic”. What are these? They work in strange ways, but no combination seems to get the jack to work at all. Something is not the way it should be.

2.2 The modem. :( NO SUCCESS as of April/November 2008

cd /tmp

wget http://132.68.73.235/linmodems/packages/scanModem.gz

gzip -d scanModem.gz

chmod 777 scanModem

./scanModem

more Modem/ModemData.txt

So maybe it's a si3054 modem?? Then installed http://packages.debian.org/lenny/amd64/sl-modem-daemon/download

to get result:

: Selecting previously deselected package sl-modem-daemon. (Reading database ... 171513 files and directories currently installed.) Unpacking sl-modem-daemon (from .../sl-modem-daemon_2.9.9d+e-pre2-10_amd64.deb) ... Setting up sl-modem-daemon (2.9.9d+e-pre2-10) ... md5sum: /etc/udev/sl-modem-daemon.rules: No such file or directory Adding system user `Slmodemd' (UID 115) ... Adding new group `Slmodemd' (GID 128) ... Adding new user `Slmodemd' (UID 115) with group `Slmodemd' ... Not creating home directory `/'. Only access through ALSA is available on amd64 but slamr driver was chosen! Make sure that an ALSA driver for your chipset is available and is loaded and that access to SmartLink modem components is supported by it. invoke-rc.d: initscript sl-modem-daemon, action start failed.

Then I wrote:

: sudo slmodemd -c CANADA --alsa hw:0,6

to create the modem...?

Then I edited:

: sudo /etc/init.d/sl-modem-daemon start

to change “auto” to hw:0,6 for modem, and country to Canada,... then

: sudo /etc/init.d/sl-modem-daemon start

and efax-gtk... It thought it was actualy a modem at this point, but it didn't work in use.

2.3 The Zareason under Ubuntu 8.10

Now, after upgrading to Intrepid / as of Oct 2008:

2.3.0.1 ubcsecure

I can no longer log on to a WPA2 wireless network. I've got all the settings right. It worked under Ubuntu 8.04.

2.3.0.2 Loose screws

two screws fell out the bottom, from the panel I described as squishy. The panel is now really popping out. Zareason says they are sending me replacement screws.

2.3.0.3 Broken DVD drive

The CD/DVD drive died. Zareason says they are sending me a replacement.

2.3.0.4 CPU speed

I have a computation process going full-tilt, using one processor at 100%. Yet the CPU frequency is only at 60%. Why? I am forcing both CPUs to scale up to their max, hoping this will speed up the process:

: cat /sys/devices/system/cpu/cpu0/cpufreq/scaling_available_frequencies
sudo /usr/bin/cpufreq-selector -f 1660000

They seem to be at 54 C, not too hot...?

Maybe safer is:

: sudo cpufreq-selector -g performance

I don't understand why my CPU is no longer throttling up automatically when it should I seem to have to use this command whenever I'm cranking. So, put following in .bashrc for frequent use..

performance()

: {

sudo echo Max speed for Zareason under 8.10...

sudo cpufreq-selector -g performance &

}

3. Favourite (open-source) software

All I need in life for productivity software is the equivalent of Matlab, Adobe Illustrator, LYX, and Maple/Mathematica. Scilab may be an open source excellent alternative (not compatible) to Matlab [Actually, I think extensions (matplotlib) to Python may be a great alternative to Matlab? and easier for data manipulation], and OpenOffice Draw, Scribus, and Skencil each does some of what I need from Adobe Illustrator, but not well enough and not together. xara? If Illustrator was sold for Linux, I would get that. I have licensed versions of Maple and Stata for Linux. Below are comments. It's sort of a dateless blog I suppose, in the modern language.

Proprietary license software is in a separate section, below.

3.1 Standard apt-get install line upon new OS install

Here's a list of standard favourites, vaguely up to date (but not complete) for Lucid Lynx 10.04: (emacs, latex are implicit in the following...)

apt-get install lyx simple-ccsm auctex offlineimap wine vlc mplayer ubuntu-restricted-extras python-mode python-matplotlib ipython htop gnome-schedule openssh-server openssh-blacklist feh sshfs curl apache2 meld sbackup gnumeric tth latex2html pdftk latex-xft-fonts texlive-latex-extra chktex dvipost inkscape gimp meld traceroute enscript antiword biblatex python-scipy iotop sysstat python-rpy bibtex2html kbibtex jabref powertop rubber biblatex-dw

It seems my python/Stata code now also wants R and rpy, hence some of the latter stuff, above

Other new install tasks:

Do not install alpine. Make a symbolic link to my own compiled version, in /home/local/. Or, if that's not an option: Compile alpine with non-open maildir patch (Argh!!). See my wordpress page on this.
Add the Medibuntu repository: need to do this every version (could just copy files from old /etc/apt/sources.list.d ). Something like this, e.g. for lucid lynx:

sudo wget http://www.medibuntu.org/sources.list.d/lucid.list --output-document=/etc/apt/sources.list.d/medibuntu.list
sudo apt-get update
sudo apt-get install medibuntu-keyring

And then install things like: (huh? Didn't find skype in 10.04 this way. So install from skype.com... hm)
apt-get install skype
Maybe want to install google repository, too (hm, easiest way to do this is just to install google chrome beta?? which automatically adds the repository and keys??), or:?

# Google software repository
deb http://dl.google.com/linux/deb/ stable non-free
Replace my customisations into beamerbaseframe.sty (or, since I've not just put a copy into my own texmf tree, you might check that the version of beamer hasn't been updated. If so, merge (meld) the changes.
I also need to set my offlineimapIfGmail.sh script to run every 2 minutes (using gnome-schedule as crontab interface).
I also need to set my backup scripts: one calls rsync if my laptop is at work, and one calls sbackupd if my laptop is at work.
Set up default web server root in default server
install commercial version of virtualbox
some minor setup for sshfs?: edit /etc/fuse.conf and add self to fuse group? a
Set up the fantastic “ufw” firewall (see Ubuntu documentation):
- sudo ufw enable
- sudo ufw allow ssh
- sudo ufw allow http
- sudo ufw allow smtp
Install virtualbox. See their web page. I use the non-free version to get USB support. Just add a virtualbox file in etc/apt/sources.d ...
To change the default paper type (what!? For ten years I have been reporting bugs to GNU distributions which don't set the paper type to the geographic locale's. Use this:

sudo dpkg-reconfigure libpaper

3.2 Zotero, MendeleyDesktop, Referencer, jabref, ...

Which one to use?!

3.3 Inkscape

Dec 2007: Inkscape is lovely, though it does not really embrace EPS format. When starting from scratch, use this. Built in tutorials are great.

Well, actually scribus can import EPS nicely, with the exception that it turns fonts into polygons, and then can export to SVG, which Inkscape reads natively.

Development seems active in this area, so Inkscape should read EPS soon.

3.3.1 June 2008: png in inkscape to .eps output broken: So...

% For some reason, saving from inkscape as .eps gives garbage for one

% of the transparent background constituent images. #:(( But I need to

% use .eps because of the .tex's I'm including (Aghh). Converting

% using command link imagemagick from .svg to .jpeg to .eps makes

% something that is much faster to display in .pdf and which doesn't

% have the bugs.

3.3.2 Bounding boxes

Didn't I figure out how to automate inkscape to do boudning boxes? yeah.. see matlab/eps

3.4 Emacs

Of course, my favourite software. March 2006: I used the Customize feature in the pull-down Options menu to change backup-behaviour. Emacs now keeps old versions of files all in one place, and keeps only a few versions (under Ubuntu it seemed to be defaulted to dozens).

3.4.1 Merging versions of text files

Why are sdiff etc so poorly documented? Well, emacs' emerge (M-x emerge-files) does nicely; see docs at nongnu.org. This is useful for merging forked versions of code. [2008: oh, no! it's not. Use meld, a lovely gui for diff'ing and merging!! Polished.]

3.5 Backup software and remote file systems

Feb 2006: Searching for good simple backup manager software. Have been using my own Perl script for some years. Found simplebashbu and backup-manager (under Ubuntu? good!) and Ubuntu's “sbackup” (GUI). I am trying the latter. After installation, access through Ubuntu menu system. After recent [2006] improvements to it, the auto SSH works beautifully. It is supposed to erase old files...

The SSH feature seems fussy. A working format for me is:

: ssh://cpbl:pw@myhost.com:/home/cpbl/private/backup

So, sbackup makes backups, and the following entry in crontab keeps only recent ones (the backups happen at noon, 00 12) and copies them to a remote machine:

MAILTO=cpbl

50 11 * * * find /backup -mtime +6 -exec rm -f {} \;

59 12 * * * rsync -vzr --delete -e ssh /backup grad.econ.ubc.ca:private/backupRsync

See the next section for remote method.

May2006: I'm not sure I like sbackup anymore, as bugs aren't getting addressed. Ubuntu seems now to be suggesting keep or backuppc; look at those...

Oct2006: No, sbackup is fixed and active.

Jan 2007: sbackup stopped working in December. It seems to crash when an old backup target directory is empty (and why is it empty? Also sbackup's fault? well, any failed backup crashes future backups)... yet another bug report. Apparently the author is too busy finishing up his Master's.

3.6 ssh / scp / rsync: automated authentication

btw, to make automated ssh/scp/rsync etc (ie so above process doesn't need a password), I use the following (this makes it work both directions; only one is desirable) with a blank passphrase...

Added same users on each machine

typed ssh-keygen -t rsa as that user on “from” machine. Chose no password (ie press enter)

connected from “from” machine to other via ssh ( typing password ) which makes ~/.ssh/known_hosts

copied from “from” machineA ~/.ssh/id_rsa.pub to “to” machineB ~/.ssh/authorized_keys (need to concatenate, or cut/paste, if authorized_keys already exists!)

Presto. You never need to enter a password anymore to ssh or scp to the “to” machine from the “from” machine. Insecure? Not really.

3.7 autofs

Outstanding! I don't know how old this is, but this is probably the way to mount all sshfs, cifs, etc mounts. E.g. no more worrying about whether my backup scripts are writing to a local disk because I haven't mounted a cifs one.

Downside: but the reason these things weren't in my fstab is because I don't want to hose the network when I'm not wired (to the network). So I really would like to be able to insert some conditions / scripting in the autofs process..

Also, what happens when it's not available, or when the network connection breaks/changes?

3.8 sshfs

May2006: Oh! The obvious has been invented. Simply mount a foreign filespace through ssh!

: sudo apt-get install sshfs

and then add user “cpbl” to the group “fuse” in order to give me permissions to mount stuff...

: sudo adduser cpbl fuse

Also, in order for root to be able to access an sshfs volume that a user has mounted, uncomment the line “user_allow_other” in /etc/fuse.conf and then use the “-o allow_root” option in any sshfs call.

So, [In Feisty I needed to log out and back in after installation, but not in Gutsy] in a startup script run:

[actually, in Dapper, I seem to have to do a “modprobe fuse” or to put fuse in /etc/modules in order to get /dev/fuse to exist... This may be a bug.] [Not necessary by Gutsty].

: sshfs -o idmap=user -o allow_root cpbl@remotehost.org:mydir/ /localmountpoint

where localmountpoint is owned by the user. Absolutely brilliant. Backups just go straight to the server now.

The idmap flag is making it so that the remote user's uid looks like the local user's uid. This is optional. The allow_root flag seems to allow root to write to the mounted volume.

[ql

Existing documentation is weak. So altogether, this seems not so user-friendly yet. There should be a gui to set up mounts and offer to make them permanent, etc. ]:

[June 2006] An upcoming version will allow fstab entry for sshfs. However, that is not an advantage. Because I have public keys set up for the remote host already (see above), I can mount the remote system entirely automatically by putting it in a (e.g. Gnome) personalised startup option (through gui). Doing it as a user, not root, means no password is needed. But it means that I have to log in for it to be there, which is not great for root-running backup scripts... It seems that Matthew Hoskins (google) has come to the same conclusions as me. He mentions that this method does not cleanly unmount the volume at shutdown..

Note also that if you don't need command line, you can use Gnome or KDE file browsers to “connect to a server” by ssh; they'll stick the folder right on the desktop and you have gui / drag and drop access.

3.8.1 Connection dropping

[The following site-specific problem resolved itself in mid 2007]: For some reason at AERL building at UBC, but not at GC or by wireless, my sshfs connections get dropped (especially if idle for some minutes). If I understand SSH has anti-idle forcing abilities. I could put these on the command line in sshfs with -o, or just put “ServerAliveInterval 15” into my .ssh/config file. I did the latter. Now network is being flakey in general from AERL. Does having a steady ping session going help keep things alive?

3.9 Laptop / screen presentations (slides) and PDF, L^ATEX, LYX.

Trying to choose a format to use for the next few years for overhead (projected) presentations. Requirements:

if material does not fit on a page, it does not disappear; an extra page gets created (like the ancient slitex slides.sty).
- Preferably, the title would be (optionally) repeated on these extra slides, with a “cont'd” suffix or etc.
- The point is that I'd like to be able to use an existing (e.g. article) document and convert it to slides very easily.
I'd like to be able to use LYX with it.
I'd like to be able to use not-so-huge fonts, and turn off all glitz, but
it should generate PDF bookmarks/TOC etc, and be able to give PDF directives like “fullscreen” when compiled to PDF
some smart image sizing would be nice

LYX supports seminar and foils (which requires manual installation of L^ATEX class), but seminar fails on the first criterion. If the first criterion is met, then at least the thing can be edited in LYX and then at the end converted to a screen presentation with minimal difficulty (since “\section” can become “\slide”.

So I am [was, 2005] following Matt Welsh's advice, even though — UGH — it involves learning about true-type fonts. It involves some investment in setup, but looks like what I want.

Why is foils.cls so hard to find, and not installed? Why does LYX not carry it any more? Oh.. It seems foils might actually do what I want... so I should get LYX's template to work (someting's wrong!): Aha: Foils has a restrictive license, so tends not to be included in free L^ATEX distros. See “Latex installation” under the LYX “Help” menu .

Hey, under foils, just modify title environment so it also sets the top-right header to the title, and suppresses the top-right header for the current page! DONE! Ask me for cpblFoils.sty if you want this modification!

3.9.1 Summary of available options

Okay, based on Michael Wiedmann's standard catalogue of options for L^ATEX-based presentation packages, here is some better overview info.

From my point of view, any package which will eliminate material which does not fit on the page defeats much of the charm of the L^ATEX solution; that is, one must worry about form (how much will fit on a given slide) when one should focus on content. This narrows down the package options:

The following require “begin slide” and “end slide” tags of some form. In general, these seem to be able to look much snazzier, but that doesn't help me:

slidenotes
beamer (nice!)
HA-prosper / prosper / seminar / ifmslide (HA-prosper has impressive look, indexing, commenting)
pdfscreen

The following allow one to start a new, titled slide whenever one wishes, but will display on extra pages any material which does not fit on a given page. These are therefore candidates for my use:

texpower:: an add-on package; great
foiltex:: fundamental for most of what I like so far
PPower4:: this is an add-on also; great
rayslides:: this looks good too! check it out?
texslides:: Matt Welsh's page collects some of above and deals with font issues for PDFL^ATEX

I now still have to figure out any font (PDFL^ATEX vs L^ATEX) issues, and optimise compatibility with LYX.

3.9.2 Conclusion so far:

LYX with foils class with my add-on cpblFoils.sty does everything I want! texpower would likely do any extra glitz.

: \usepackage{cpblFoils}

Making dvi when hyppref is used causes errors. (Oh? I think everything works perfectly using PDF not PDFtex in Lyx – nov 2005).

I think I have the best of both worlds now...

but: the font for math is a bit odd?

-it would be nice to have PDF bookmarks/toc generated for foil headers.

sept 2005: There are some other good tricks, like using the “shortform” option for drafts, and making use of the comment package. See my slidesTemplate.lyx: It can switch to make a compact version of the slides with comments included. (In LYX, the “shortform” option goes in the Layout $\rightarrow$ Document Style $\rightarrow$ Options box)

Nov 2005: But this foiltex does not allow use of bibliography / bibtex with slides. Why? And I can't find discussions of this. Needed modifications so far:

simplify comment switching on/off?
modify LYX layout and L^ATEX style to allow bibliography and references usage
allow use of sections, subsections, etc, etc rather than foilheaders: Have sections behave as foilheaders?? This makes turning an article / thoughts / paper into presentation easier?

When I am desperate to tighten up a list, I have used “\zerolistvertdimens” at the beginning of the list.

2007: I now use Beamer. Interface within LYX is okay, though I've retreated to L^ATEX for Beamer.

3.9.3 Beamer

See the separate section on Beamer, since I have chosen to go that route.

3.10 Beamer (L^ATEX/LYX)

May 2007: It does not meet all my requirements (hm, actually, you can set a single frame to continue if over-full. maybe the whole document?), but I'm thinking of switching to .

In LYX, it seems I need to manually set the page size to 96 $\times$ 128~mm for reasonable output!

Emacs: Whizzytex is interesting (view output as you write), but I can't get it working with beamer. I'm also not sure that I've got auctex / beamer working to full effect in emacs.

Themes: boadilla or singapore.

May 2007: At least with LYX 3.4, various things do not work with Beamer, e.g. pdfpages. So I have switched back to good old emacs and latex. pdfpages still does not work, though at least it compiles. The included page appears but then gets overwritten with blankness immediately. With beamer, in each slide you can choose between having continuing frames (ie material not lost if it overruns a page) or having available overlay features. A complete manual for beamer is included on-disk.

Beamer works in either ps (eps graphics) or pdf (pdf graphics) modes, which means I can use figures made with Laprint in Matlab!

2011 Oct: I've been using nothing but Beamer for years now, though rarely/never in LYX.

3.10.1 November 2008: full-screen images

I've made, in my cpblRef.sty (available on software page), a family of commands for display of a graphic that takes up the whole screen, or the whole screen but respecting Beamer's margins, or the whole screen except for the title and margins, without messing up the aspect ratio of the image.

The calls just look like:

\plainFrameImage{anImage.jpeg} % respect margins

\maxFrameImage{anImage.jpeg} % don't respect margins: use entire screen

\titledFrameImage{My frame's title}{anImage.jpeg} % respect margins, and include a title

Actually, they are imperfect. They also rely on a small modification to one beamerbaseframe.sty style file; see comments in my cpblRefs.sty.

3.10.2 What about using the same file for beamer and articles

Mostly, I've given up on this. But for a course, I want to have the syllabus TOC appear in an article, while the slides are made into beamer class. Shall I redefine some “subsubsections” into slides?

Python script?

Actually, see the beamer manual for its beamerarticle.sty to use under article.cls. But currently I have a problem with this, due to the modification I've made for full-screen images (above).. I have to iron that out. I'm using the handout mode just now. (Jan 2009).

3.10.3 Bibliography

2011 Oct: I think it may be best not to use BibTEX in Beamer. Just include things by hand. I can't remember why (but I think it fails somehow).

3.11 LYX

Under “Help”, LYX gives info on current L^ATEX installation. Nice. Followed its advice to install cv.cls in my user space. I ought really to have installed foils in my user space, since it doesn't come with open source. [I think I have moved from Foils in LYX to Beamer in L^ATEX – 2007, though clearly some people do alright with Beamer in LYX]

For making tables, if you want to have horizontal lines per cell, turn “multicolumn” on individually in each cell.

3.11.1 Macros

Macros in Lyx are a pain. You cannot define commands outside math environment. I guess I failed to get the template thing working properly!? So here was my solution: I put the latex macros into one file (in the appropiate place in ~/texmf/tex) and redefined them for lyx in another file.

: alias lyxm='lyx ~/latex/lyx/lyxMacros.lyx '

Actually, this needs to load the arg before the lyxMacros.lyx file... so in bash it's a function, not an alias; from my .bashrc file:

lyxm()

{

lyx ${1} ~/latex/lyx/lyxMacros.lyx &

}

Or, alternatively:

Load the lyxMacros.lyx file first into Lyx. Within this file, note that for things to work, the order of definitions matters. my alias expect must be before cexpect, for example. This tells LYX, NOT L^ATEX, how to display defined macros.
Now load the file you want.
put a

\usepackage{cpblEcon}

in the documnet preamble for LYX (can set this as part of default). And put a latex file cpblEcon.sty in the latex path (e.g. ~/texmf/tex/latex/) with the appropriate L^ATEX definitions corresponding to the LYX ones, above (!).

Note: I have also written a LYX style file. It has a \usepackage{cpblEcon} in it, and defines some new environments.

Feb 2005: Ack. a bug in Lyx? Can't get macros which use mathbb to work in foils. works when exported to latex. :( Solution: including “\usepackage[cspex,bbgreekl]{mathbbol}” seemed to solve it.

Nov 2005: As general policy, put new style bits into the “preamble” sections of lyx layout files or .inc files for layouts, rather than having separate L^ATEX style files... I'm making comment, foils, and econ paper styles.

3.11.2 Bibliography / BibTEX

In Document layout, choose natbib: that way, you can use the different kinds of citations, e.g. noun or parenthetical etc.

3.11.3 URLs in HTML output

Please tell me how to get them to work. LYX uses latex2html to make its html output, and hyperlinks do not work.

I read somewhere (but it doesn't help):

*All* documents that make significant use of hyperlinking should \usepackage{html} when being translated into HTML using L^ATEX2HTML. That is a general rule for which there are no real exceptions, in normal L^ATEX usage.

3.11.4 Inserting PDF documents / pages into your latex file as full-page figures/tables, etc.

If you're using pdflatex (not latex), then you want the package pdfpages. Brilliant.

3.12 L^ATEX

How do I make an underbar that looks just like \bar in math mode? The closest is the ushort.sty package, but its \ushort is not the same width nor thickness as \bar.

3.12.1 letter size

How is it possible that in 2007 it is (still) necessary to put the following line

\usepackage[pdftex]{graphicx}

in order for pdftex to use my local paper size (letter) rather than a4?? (or is that just with the resume package?)

May 2010: Indeed, how can it be in 2010 that the same problem exists?! And the above line doesn't work for me now. Using

: \usepackage[letterpaper]{geometry}

is the only thing that did. (Ubuntu 10.04).

2012: After upgrade to 12.10, I found that including geometry package without the letterpaper option gives A4. I just tried on 12.04 and it seems using geomtry doesn't mess up there.

3.12.2 General

I have a separate web page on L^ATEX lessons. (Search/google for latex and CPBL).

I put some L^ATEX style files from texslides in my texmf directory (apparently the structure here has to mirror that of the system texmf dir??) and used

: texhash

to update the database. To see where things (personal .bib database, .bst files, etc) are meant to go, use, for example

: kpsewhich -progname=bibtex -show-path=bst

You should not need to set the environment variables to choose paths. Another neat program is texconfig, run as root, to set latex settings.

3.12.3 How to convert L^ATEX to HTML

I really do not know.

latex2html [does not use mathML markup, but makes little images for all math elements: it's long out of date]
tex4ht (htlatex), [sounds great but appears not to process my macros at all]
hevea [reads latex but not tex, so can't parse pure tex style files]
tth : this seems to be what LYX uses by default (jan 2008).. is it thesis capable?

all fail broadly on my thesis. Though it seems that I have tweaked latex2html to work partly in the past ...

2012: update: The following seems to produce html with macros substituted, etc, etc!

: htlatex myfile.tex

See

http://www.tug.org/applications/tex4ht/mn-commands.html

for more, e.g. oolatex (resulting .odt made libreoffice barf!!) or one for MS word..

2012 update: what about pandoc?

3.12.4 Tables

Tables ! A lack of a great GUI latex table editor is a problem. Tables seem the reason not to use L^ATEX, and also the reason not to use the alternatives (office suites!?. Yuck!)

There are many packages that attempt to do tables now. Make sure to use the array package, which modifies tabular environment to be a bit better. Other than that, I'm now just using tabular and longtable. I'll have to have a look at tabularx, too. My python code which makes tables from Stata output using landscape and longtable depending on the size of the table.
However, I have abstracted from particular packages by making a standard format of defintions for any table to include. By varying the name of the function I call a table file with, I can invoke different behaviour and different packages for the same table tex code. So my wrapper functions include:
- \cpblTableBSimple
- \cpblTableBWide
- \cpblTableBLong
- \cpblTableBSimpleLandscape
- \cpblTableBWideLandscape
- \cpblTableBLongLandscape
Their arguments include the name of a .tex file which defines certain standard parts of a table. The code is all available on my web page.
How do I centre a table which is wider than the textwidth?
How do I make a landscape table appear at the beginning of a new page? Since one calls the landscape before calling the float creation, landscape tables appear immediately where you call them, which is of course impossible and contrary to the point of L^ATEX.
For long landscape tables:

\usepackage{longtable, lscape}

and use the table with

\begin{landscape}
\begin{longtable}{ccc}
\caption{Title of your table} \\
Header of first column & Header of second column \\
\endhead
Table cell 1, 1 & Table cell 1, 2 \\
Table cell 2, 1 & Table cell 2, 2
\label{label-name}
\end{longtable}
\end{landscape}
Here's a horrid trick for desperate times! Just includ graphicx package and use the \resizebox command: it works on any text object (e.g. a table) too.:
\resizebox{!}{10pt}{\begin{tabular}{|l|c|} \rowcolor[gray]{.9}[0pt][0pt] one&two\\ \rowcolor[gray]{.5} three&four \end{tabular} }
[2008] Coloured cells in tables: \usepackage{colortbl}. Beware that the released/public version has a limited and problematic functionality with cellcolor. It assumes that cell spacing is \tabcolsep. Thus, the following works,

\setlength\tabcolsep{1pt} %6pt is default!
\begin{tabular}{|l|c|c|c|} one&two& three\cellcolor{cSignifOne}&four \end{tabular}
\setlength\tabcolsep{6pt} %6pt is default!

but in this version the colour overruns the cell badly:

\usepackage{colortbl}
\definecolor{cSignifOne}{rgb}{.92,1,.92}
\begin{tabular}{|l|c|c|c|} one&two& three\cellcolor{cSignifOne}&four \end{tabular}

So, here are some useful incantations:

\usepackage{colortbl}
\definecolor{cSignifOne}{rgb}{.92,1,.92}      \newcommand{\greenbox}[1]{\begin{tabular}{@{}c@{}}\rowcolor{cSignifOne}[0pt][0pt]#1\\\end{tabular}}

and possibly using multicolumn one could make a better version of the cellcolor command. (By the way, its author tells me that he his old version of cellcolor takes a spacing argument, but this is not the case in the standard package).
Here are two more ideas from usenet: you can either overwrite the overhang with a \columcolor or \rowcolor-command or set \tabcolsep to a smaller value:

\renewcommand{\green}{\cellcolor{cSignifOne}}
\begin{tabular}{|c>{\columncolor{white}[1pt][1pt]}cc|} one&\green two &three\\ a & b & c \end{tabular}

and

\setlength\tabcolsep{1pt} \begin{tabular}{|c@{\hspace{2pt}}c@{\hspace{2pt}}c|} one&\green two &three\\ a & b & c \end{tabular}
I have encapsulated a bunch of what I have learned about tables (much updated from the above) into my set of formats cpblTables.sty (see my software page). I write tables to a .tex file in such a way that I can just use a different call to customise the display format based on the length and width of the table.
I also sometimes use editLatexTable.py (see my software page) to separate table elements into spreadsheet cells for each cutting, pasting, and switching on and off of rows and columns.

3.12.5 Font sizes

I have .tex files containing \tabular environments that I include in both an article (full size) and, for presentations, in beamer (fonts are very small). I have set it so that some text I need small uses a \smaller command, but that does not work (using {relsize} package) for fonts smaller than the predefined ones! (how annoying! relsize should just make an absolutely smaller font!). So I've had to use an ugly technique of specifying font size exactly by redefining a command as needed, e.g.:

: \newcommand{\muchsmaller}{\fontsize{4}{6}\selectfont }

3.12.6 Laprint (see Matlab)

3.12.7 Random

3.12.7.1 ifpdf

One of the main tools is to use the ifpdf latex style. This enables you to choose the correct latex packages when you start running (pdf)latex through your files. One use here is to choose the correct graphics driver:

\usepackage{ifpdf}

\ifpdf

\usepackage[pdftex]{graphicx}

\usepackage{epstopdf}

\else

\usepackage[dvips]{graphicx}

\fi

3.12.7.2 epstopdf

Just include the epstopdf package in your latex document and invoke pdflatex with the –shell-escape option:

pdflatex –shell-escape test_document.tex

but thats it ...

3.12.7.3 \newif

\newif\ifsolutions\solutionsfalse

this defines a conditional \ifsolutions which can be used in the document. Here this conditional is set to false. In order to use in in the text, do this

\ifsolutions

% Put text to be shown if solutions is true here

\else

% Put text to be shown if solutions is false here

\fi

3.12.8 Troubleshooting

3.12.8.1 2008 May: latex refuses to write .aux file

: pdflatex: Not writing to /home/cpbl/somefile.aux (openout_any = p).
! I can't write on file ...

Solution: use a relative pathname, not an absolute one to \include .tex files! (I am replacing all pathnames with macros so they're easily changed.) This is some security feature of L^ATEX's.

Sept 2008: Ah, but it can be overruled by changing an environment variable before compiling!:

: export openout_any=r

3.12.8.2 Nightmare with hyperref, prettyref, and pdflatex

September 2008:

Using ubcthesis class and hyperref and pdflatex, I got horrible errors like:

: ! Missing number, treated as zero.
<to be read again>

along with strange double page numbers at a chapter boundary.

The solution ended up being to write my own prettyref shorthands and to get those definitions set in the right order with respect to the usepackage{hyperref} and to completely ditch prettyref.

3.12.8.3 includegraphics file not found: L^ATEX gives confusing error for graphics files included with an underscore in the filename.

Feb 2010: Agh! Files that used to compile don't anymore because L^ATEX now fails to find a file if it has underscores in it. I could not find a solution (using the underscore.sty package didn't solve it) so I have renamed countless graphics files to have hyphens in place of the underscores. Why can I not find this problem on the 'net?

Dec 2010: aha. It's because an include was including the package “underscore”. Contrary to my comments there, this seems to cause (not solve) the underscore in filenames problem! See documentation for underscore.sty, where this incompatibility and some workarounds is clearly explained. for me, using [stirngs] or babel does not seem to help. Overall, avoiding underscore.sty seems the way to go....

3.13 Python

3.13.1 Regexp editor (Kodos)

If ever you need to use any regular expressions, use Kodos to develop them! Brilliant.

I tried some Python IDEs. The best I tried was SPE (Stani's), but it ground the computer to a near halt. In the end, of course,

apt-get install python-mode

turned emacs into the tool I wanted.

3.13.2 ipython

For development, debugging, always run python programs in ipython. Put the command '1/0' anywhere you want to “break” into a debugger, and just type pdb once in ipython when you start up.

3.13.3 Plotting like Matlab in python

agi python-scipy python-numpy

This stuff is maturing wonderfully, and coalescing into SAGE to replace all of Matlab, Maple, Mathematica, etc...

The plotting in python is improving and already the best thing out there, it seems.

To make it so that plot figures are not modal, ie the session is “interactive” in pylab's lingo, ie execution does not wait for you to close a figure every time a figure is shown, set interactive: True in matplotlibrc?

Then, to autocrop the resulting image, can use imagemagick: see my .bashrc “autocrop()” function.

July 2010: I no longer use Matlab (maybe except for maps?). Need to write here a section on pylab figures in L^ATEX!!!, bounding boxes, etc.

3.14 Open source GIS software

3.14.1 Plotting maps (and reading .shp shape files) in Python

I need a replacement (hopefully better/faster!) for Matlab's mapping toolbox, to use with pylab/python.

2008? Mapping toolbox: Need to install the GEOS library which comes as part of the python matplotlib add-on “basemap”. Can read .shp files, make maps, etc: a replacement for Matlab's mapping toolkit.

2009 Sept: It seems there are a host of GIS packages for python in the works. None seems to work reliably on a broad set of .shp files. There are efforts to consolidate many of these, but they're still in progress! What is to be done, then?

I was using basemap to plot, but it's automatically installable under Fedora but not Ubuntu. But it can't read my shape file (in lambert coordinates). osgeo seems more promising. It installs automatically on Ubuntu but not Fedora!! What is the best ?

http://www.scipy.org/Cookbook/Matplotlib/Maps shows that basemap is mature and even comes with good data! My needs are to make country/global maps and use canadian boundary data to make within-Canada region maps. So (sept 2009) it looks like this is becoming good. Sadly the demo shown there gives a seg fault on my installation! But a modification works.

Apparently a utility can convert a .shp file to lat/lon, but it fails for me. “You can convert the shapefile to geographic - coordinates using the shpproj utility from the shapelib tools - (http://shapelib.maptools.org/shapelib-tools.html) “unable to process projection, exiting...”

I have now taken the extraordinary step of using Matlab to load the shapefile, then write out its data in my own text format, and read that into python in order to be able to plot things using simple plotting of patches...

Well, after much time trying, Duncan told me to stop scripting and just use Quantum GIS (qgis). I found it on Fedora but not on Ubuntu, at first... and am having a hard time getting it to do what I want so far. For Ubuntu someone advises:

: apt-get qgis grass qgis-plugin-grass mapserver-bin gdal-bin cgi-mapserver install python-qt4 python-sip4 python-gdal python-mapscript gmt gmt-coastline-data r-recommended gpsbabel shapelib

as well as libgdal1-1.4.0-grass, which failed for me. And so did the qgis ones... So stuck with using the Fedora machine and banging my head against qgis...

3.15 Mplayer / Playing video

Mplayer sounds like it is (again?) the cutting edge video player. For watching interlaced video (high speed motion shows combs if player is showing both halves) you can try to construct full images by CPU-intensive blending. “mplayer -pphelp” will show some available methods; you just try one. For example, “mplayer -vf pp=fd:c” seems remarkably good.

3.16 Rotating and editing video

2006/7: I've a digital camera now, so I need to easily rotate video files, and sometimes to cut sections from them. What's the right tool? The non-avidemux way to rotate by +90 degrees into the page is:

: mencoder -vf rotate=1 -ovc lavc -oac copy mov00328.mpg -o ronaldo.avi

3.17 PDF

: apt-get install pdftk flpsed

There is still [2006] no good GUI editor of PDF / fillable forms!? Amazing. Adobe does not offer any decent software for unix for some reason. Anyway,

in order to crudely but easily add text to PDFs, run “flpsed” and import/export the PDF. There are other similar bitmapping methods to this.
in order to merge pdf files:
e.g. pdftk file1.pdf file2.pdf cat output merged.pdf
or use guipdftk for a graphical interface.
in order to fill out forms cleanly (I haven't done this properly), use pdftk with the fill-form function.

Update 2007 March:

Install “alien”. Then download the latest rpm for “pdfedit” and use

: sudo alien -iv <rpmfile>

I haven't yet figured this out, but it looks really promising.

Update Feb 2008: It's now integrated in Ubuntu: easy installation. But still buggy.

May 2008: But the following beauty seems to work fine under wine!!

: http://www.docu-track.com/download/PDFXVwer.zip

3.17.1 Extracting a chart/figure from a PDF to include in something else

The following is in my .bashrc:

# Convert one page of a pdf file to a graphical format at high res (ie extract a figure or table)

pdfExtractFigure() {

pdftops ${1} - | psselect -p${2} > tmp_p${2}_${1}.ps

echo Use 300 dpi resolution and B/W mode to import this file!

echo Then drag and ctrl-x regions to eliminate, and then use image->transform->autocrop to get what you want. Save as eps. (But do you really need to? Just skip between files when you want a table: in Adobe, use Ctrl-L and Ctrl-2 to make a page screen-width.

gimp --no-splash --no-data tmp_p${2}_${1}.ps &

}

3.18 Fax modem

May 2006: I thought I'd try sending a fax. I quickly determined to try

apt-get install efax-gtk slmodem

The application efax-gtk is brilliant. It sends any set of postscript files by fax using the softmodem in my laptop, with perfect interface...

2007 March: I had difficulty sending a multi-page PDF which was cobbled together with pdftk. Better to tell efax the various files as separate files.

Nov 2007: Under gutsy, don't need slmodem. Just install efax.

3.19 GIMP

Actually, GIMP is not a favourite software.. well, maybe becoming more so.

But: to make an image background, or one colour, transparent: really easy now. Use Layers $\rightarrow$ Transparency $\rightarrow$ Add alpha. Then use Colours $\rightarrow$ Convert colour to transparent. This will probably guess the right colour automatically: done. You can also use the magic wand to select a region once you've added an transparent layer.

2008: Now I've made a script so that I can automate some transparency from the command line. In ~/.gimp/scripts is a file:

(define (simple-white-to-transparent filename

filenameout

)

(let* ((image (car (gimp-file-load RUN-NONINTERACTIVE filename filename)))

(drawable (car (gimp-image-get-active-layer image))))

(plug-in-colortoalpha RUN-NONINTERACTIVE

image drawable '(255 255 255) )

(gimp-file-save RUN-NONINTERACTIVE image drawable filenameout filenameout)

(gimp-image-delete image)))

and I use it as:

: gimp -i -b '(simple-white-to-transparent fromfile.eps tofile.png )' -b '(gimp-quit 0)'

btw, a good image viewer is: feh

Sept 2008: Another script: to autocrop figures. and associated “prepareFigures.py” has been started.

May 2009: Actually, I just have a bash script in .bashrc now for the autocrop:

autocrop() {

# Could I do this better with mogrify??

for g in $@ {

convert $g -trim +repage ${g}_cropped.png

} }

3.20 OpenOffice

One of my *least* favourite software, not most favourite – ie it baffles me.

I installed calc2latex macro – maybe use it next time.

3.21 Batch conversion from Microsoft proprietary formats

3.21.1 Batch conversion of office document types (Freedom from Microsoft)

Brilliant! : just load this macro into OpenOffice, and you can turn all your .doc files into .html or .pdf or any other combination imaginable – text, spreadsheets, etc:

http://ooomacros.org/user.php#95532

3.21.2 batch conversion of wma to mp3/ogg (Freedom from Microsoft)

Okay, I don't have this trouble, but if you have .wma files, try:

sudo apt-get install soundconverter

wheee!

3.22 (Al)pine

Well, I rather wish Pine was open source and more functional, so this is again not “favourite software”, but it's my favourite email client. [Update: in 2007 it became open source as “alpine”.]

At UBC, the unencrypted SMTP service for Interchange email works by setting the “smtp-server” in Config options to

smtp.interchange.ubc.ca/novalidate-cert

In July 2006, the unencrypted SMTP service became flakey but the encrypted versions werent. So if I set the “smtp-server” to

smtp.interchange.ubc.ca:587/tls/user=[ubcusername]

I get encrypted sending, but I have to enter my Interchange password once per Pine session. There seems to be no way around this, which is pretty annoying.

When interchange is really slow, just use:

smtp.gmail.com/tls/user=<gmailusername>

3.22.1 June 2009

Well, Alpine IS open source now. And I have just discovered that a program imapoffline exists!! Oh wondrous thing. Also existing is a maildir patch for alpine, for some reason still not built in!? If it were, the combination of alpine and imapoffline would be absolutely brilliant. I will move to something like it very shortly...

Oh, maybe not yet.. There is some crazzzy stuff happening on this issue of incorporating maildir into alpine: http://bugs.debian.org/cgi-bin/bugreport.cgi?bug=405762

Oh, but 31 Aug 2009: It loks like Chappa has released the patch: http://staff.washington.edu/chappa/alpine/info/maildir.html

so where is the update??

3.23 Stream recording

streamripper works for some. If it doesn't (e.g for mms protocol) use mplayer:

3.23.1 For audio:

mplayer 'mms://YourStation.wma' -ao pcm

Followed by:

lame audiodump.wav myradiostream.mp3

3.23.2 or e.g. for wmv video:

: mplayer -dumpstream 'mms://bla.wmv'

and then rename the stream.dump file to infile.wmv

Afterwards, this can then be converted to any other format with something like:

: mencoder infile.wmv -ofps 23.976 -ovc lavc -oac copy -o outfile.avi

3.23.2.1

3.24 dvd ripping

To extract a segment from a dvd, dvdrip (rip, then for transcode pick ffmpeg and the output size) followed by avidemux (to easily trim out piece of interest) is smooth and easy.

2009: Similar finding, although it seems I needed to use the mpeg_ps A+V as output format to get audio with it. But sometimes it hung..

Kino worked okay too except for changing the shape (put a border around widescreen format?)

3.25 Foreign / accented characters

It's so easy in (my) GNU/Linux: just use the right Alt key followed by something that looks like the accent (e.g., comma for cedile, apostrophe for acute) and then the letter.

Update for Ubuntu Feisty: it's even better now. From your desktop, choose the system menu $\rightarrow$ preferences $\rightarrow$ keyboard. Then the Layout Options tab. There, choose which key you want to be your “compose” key. That then works as above.

3.26 Meld

That's right, a GUI version of diff/merge. Sad to admit, I never really figured out emacs' merge. This is excellent! Standard with Ubuntu.

3.27 DMG format files (hfs)

file stuckfile.dmg

$\rightarrow$ stuckfile.dmg: VAX COFF executable not stripped

sudo mount -t hfs -o loop stuckfile.dmg mntdir/

$\rightarrow$ mount: wrong fs type, bad option, bad superblock on /dev/loop1, missing codepage or helper program, or other error. In some cases useful info is found in syslog - try dmesg | tail or so

Mounting via hfs or hfsplus failed as above.

Used http://hem.bredband.net/catacombae/index2.html java to make a new file. That one was like a CD and could be mounted in different ways to find different stuff!

sudo mount -t hfs -o loop newfile.iso here/

sudo mount -t iso9660 -o loop newfile.iso here/

3.28 VirtualBox

3.28.1 Jan 2010: Ubuntu 9.10 Karmic

Huh? I installed the non-OSE (needed for USB connections) as in instructions on virtualbox's site.

But the link doesn't come up in my menus until I log out/log back in!

And starting “VirtualBox” fails wth

WARNING: The vboxdrv kernel module is not loaded. Either there is no module

available for the current kernel (2.6.31-16-generic) or it failed to

load. Please recompile the kernel module and install it by

sudo /etc/init.d/vboxdrv setup

You will not be able to start VMs until this problem is fixed.

When I try to do that, I am told that the file/program does not exist. The closest thing that does is:

: sudo /etc/init.d/vboxdrv.dpkg-bak

and when I do that, VirtualBox works again.

3.28.2 May 2008: Hardy

May 2008, under Hardy: whooa!!!!!! I cannot believe it. The future is here. I long ago deleted any Windows partition and bought my new laptop without a windows license, but I still would like to be able to use Adobe Illustrator once in a awhile, and Wine doesn't do it justice.

But I just learned of VirtualBox's opand it blew me away. It held my hand while installing my old copy of Windows and works flawlessly: I now have a full-speed, native Windows OS running inside one window of Linux. Incredibly easy.

: apt-get install virtualbox-ose virtualbox-ose-modules-generic

(and maybe some additional things that apt installer will suggest?)

Then

: usermod -G vboxusers -a <myusername>

Then just run virtualbox (VB) from the main menus, insert Windows CD, tell VB to mount it. Go through install. But the video driver was only offering low quality, small display (and presumably other peripherals weren't up to snuff).

Important step, therefore: Once booted into the guest OS, from the Guest window in VB, tell it to “Install Guest Additions” which will do lots of magic (or, in older version, will add an external disk to the Windows desktop. Run that installer, ...) and all your virtual drivers will be in place! For instance, a magic video driver that automaticall resizes, etc, etc will be in place. Unreal. Network works out of the box through VB's NAT.

I guess I can use my modem/fax etc too, that Zareason/Ubuntu has not managed to support so far.

But oops– I don't have all the fonts I need for Illustrator to edit my matlab/latex graphics... Maybe install latex??

3.28.2.1 USB under VirtualBox (and iClicker in particular)

Absolutely no problem now under Windows XP. I am teaching a course at UBC using iClickers in every class, using Ubuntu with the iclicker program running inside Windows XP under VirtualBox.

Install guest additions once booted
When VB window is not full-screened by Gnome, press right-Ctrl -F to get a full screen (no Gnome window manager decoration, no VB top/bottom menu bars – just a full screen-looking OS). Now, to show the world that actually this is just a window, press right-Ctrl and then activate the spinning cube of compiz ...
Anyway, plug in your iClicker device and bingo bingo everything works.
Also, when not booted, add a “shared folder” to link some folder in your linux path to a network share. Once booted in Windows, from the file browser, choose menu option “Tools”->”Map Network Drive”. Now you can just put your presentations there and have them handy in the windows...

Feb 2009: Problem: UBC grades synchronisation stopped working. Deleting two files (XML and .config, CMS_Integrate...) fixed this, though the uploading of grades still doesn't work.

If you need to add a student's reg data by hand, simply edit BOTH class.txt and SessionData/RemoteID.csv

3.29 Baobab: disk usage analyzer

Greeat! Find where your disk space is gone

4. Commercial application software

I am happy to buy commercial software when it's good and fosters feedback and is not an operating system. I use both Stata for Linux and Matlab for Linux. If Adobe offered better versions of its software for GNU/Linux or unix, I might buy them too (Acrobat and Illustrator for sure). I use Adobe Acrobat 8 under Wine.

4.1 Matlab

A favourite of mine.

There are references for installation — see the Ubuntu Wiki on Matlab. I have found installation etc works nicely, though I don't use the graphical IDE for Matlab programming.

Got stuck on edgy. No, fixed it.
I use Matlab over X from a remote server.
On Ubuntu 8.04, the following got rid of the reported error that occurs during startup:

: export MATLAB_JAVA=/usr/lib/jvm/java-6-sun-1.6.0.06/jre

4.1.1 nohup / backgrounding

Fantastic!! You can make graphical output simply disappear, in order to run off a server. This doesn't break even when my program gets static feedback from graphics objects (I guess they exist virtually):

: nohup \matlab -nojvm -nodisplay < driver.m >& nohupMatlab.log &

driver.m must be a script, not a function.

4.1.2 Optimisation

Typically, I start off using fsolve, and then have to switch to fmincon because I need to impose constraints. For instance, non-negativity or non-complexity.

How to do optimisations constrained to solutions in real (non-complex) space?

Using fmincon, use the nonlinear constraint realNonlcon function (in my Utilities) to ensure real.

Don't forget! fmincon minimises! Use abs() if you are trying to solve an equation.

But fmincon doesn't take vector functions, so how do I solve a vector (ie a set of equations, maybe unrelated) of equations with in positive domain...?

4.1.3 Multiple scales/axes in a plot

27 March 2007.

Different ways to do this:

some user-contributed stuff that needs work (e.g. LinkSideAxisData): Advantage: extra scale on top or on side stays with you as you rescale your plot

using “text” to place text just outside the plot border (need to rescale axes to be big first, then place text, then resize to original; that way text is not hidden in bounding box... i suppose this is taking advantage of a bug in matlab). Advantage: can use latex interpretation of the label, unlike for ticklabels.

xlabel('$\bar a = 17$','interpreter','latex');

Also, matlab has a “linkaxes” function to link different axes so they resize together!

4.1.4 EPS, L^ATEX, etc and LYX

Jan 2008. Maybe I should switch all plots over to SVG with plot2svg. LYX can deal with it very nicely, and it's open standard, editable with Inkscape, etc.?

September 2008: Here's the most promising thing in a while. Just use Matlab's crappy print to pdf, which makes a full, uncropped page.

Then inkscape –verb=FitCanvasToDrawing –verb=FileSave –verb=FileQuit <file.pdf>

4.1.4.1 exportfig

This is useful, maybe, in place of my own old “printfigure”. Use the setpref once (permanent effect), e.g.:

: opts = struct('bounds','tight','linestylemap',[],'width',6, 'fontmode','fixed', 'fontsize',12,'color','cmyk'); setpref('exportfig','defaults',opts);

and “applytofig(gcf)” to see result; then use exportfig to make an .eps.

yes, this gives much tighter bounds than my printfigure. Still, since I sometimes want tex'd text, I might need to invest in laprint at some point...

I think using 'linemode','scaled' does the nicest job: it means that differently-weighted lines ('linewidth') from the plot command are respected.

The tight bounds are cutting off my xlabel whenever xlabel is written with 'interpreter','latex'!!! Scribus can rescale the bounding box to make it completely visible, but this is a bug in exportfig.

4.1.4.1.1 Feb 2008:

I'm frustrated with exportfig. It is not giving the same thing in previewfig as it is in exportfig. I have deleted the persistent preferences (above) in my .matlab/ folder. The problem is that I have used “text()” a lot to put annotations on the figure....

It does not appropriately choose new (x)ticks so as not to have the ticklabels all run over each other
It does not result in an output that is either the height or the width specified.
It does not choose a bounding box which encloses all the text on the plot when it extends outside the axes limits.
It (or more general things in Matlab) do NOT do the same thing when Matlab is run as a batch operation using \matlab -nojvm -nodisplay as when it is run actually displaying the figures as it goes!
What junk!

Here's a trick: every text object has a property, “extent” which tells how big it is?

4.1.4.1.2 May 2008:

Another go at getting over exportfig's problems. When output mode is pdf, any latex interpreted text objects cause postscript to crash. When output mode is eps, exportfig seems to ignore the bounding box, making it loose. Yet the extent of the text object is wrongly calculated or ignored when the size of the output figure is small (e.g. 8 cm). (no, even with larger ones: xlabel is cut off at bottom, etc)

Some wonderful fellow has, however, provided some solution in the form of a Matlab contributed program called “setaxes”, along with some companions. This allows one to add extra space at the bottom. For me, it seems to mean some trial and error for each plot, though.

4.1.4.2 laprint

A beautiful tool for making nice (but EPS based!!) figures from Matlab by converting all annotations to L^ATEX overlay is Laprint.

But because it uses ps tools, I cannot use it with pdflatex. Argggh. Beamer still works with eps mode, but some PDF tricks presumably don't. beamer?

2008 September update: someone has made a magical tool, auto-...pdf, which can insert laprint figures into pdflatex runs. Wow! But for some reason I still have a low-level eps bug which plagues my system and makes this not work.

I cannot find how to make ticklabels use latex interpreter in matlab display!.

The user contributed “laprint” may be the elaborate but high-quality [but long and complex and inevitably buggy? danger!] way to go. It converts all text in a plot to latex code (!).

I still don't have a good replacement for an 8-year old version of Adobe Illustrator 8.01. (Tried to install Illustrator 8 with wine; sort of works Dec 2007; tried it under Windows 2000 inside VirtualBox May 2008: does work.)

(Tried to get laprint working with LYX. Quite difficult. Must remove language encoding to L^ATEX default...

But also, must use ps2pdf mode for viewing. (ie, dvi always looks garbage; conversion to ps is necessary). That means I cannot use this with Beamer!! No pstricks. :(

Also, must use explicit directory for .eps files in Laprint, since, LYX can't deal with .tex include files which refer to local .eps files! Since it compiles stuff in a temporary directory. I cannot see how to give an explicit eps directory from within Laprint, unfortunately. [Bug request sent to author]. Nor does it allow adding some text to the .tex file from within Matlab.

So my solution is to add a line such as the following to the L^ATEX preamble within my main, master document within LYX:

: \graphicspath{{/home/cpbl/mypath/figures/}{./}}

This is a bit restrictive; it means I cannot have any duplicate filenames amonst the set of paths that I use for graphics files.

Using this whole method in LYX reduces the utility of LYX somewhat – it means that one cannot see figures graphically within LYX. It also makes the output format less transparent, since one is committed to dvips.

Google tells me that other people are doing somewhat similar things using xfig, which will export a pair of files (as does laprint), one eps and one latex's picture environment (rather than psfrag). Each has some advantage. But I don't know of the eps/picture option for matlab.

2008 April: not merged with bove: However, the author (Linnemann), suggests this, since the alternative means having paths fixed in each .tex file produced by laprint.

4.1.4.3 2008 April: Requirements for reasonable output from Matlab:

transparency: somewhat important in eps, but also important in raster formats because colleagues using MS Powerpoint
vector and font editibility with inkscape, etc. ie. should export to SVG!!
really tight clipping/bounding-boxing that does not cut anyting off!!

I cannot even get Matlab to export as PNG without crashing right now. Export as EPS, use imagemagick “convert” to make png, and then use gimp (not imagemagick – it does a horrible job with (aliased?) edges of things) to do “color-to-alpha” ) to make the background transparent. This seems the easiest process.

SOLUTION?: April 2008:

Export as EPS. Then use a gimp script to convert background to transparent in a PNG. Imagemagick's simple transparent conversion does horribly with aliased edges, but the GIMP does well. In fact, in interactive mode it automatically selects the right background colour. I don't know how to do that in a script, but I've saved the following in ~/.gimp-2.4/scripts/makeTransparentCPBL.scm (I think the name could be anything):

(define (simple-white-to-transparent filename filenameout )

(let* ((image (car (gimp-file-load RUN-NONINTERACTIVE filename filename)))

(drawable (car (gimp-image-get-active-layer image))))

(plug-in-colortoalpha RUN-NONINTERACTIVE image drawable '(255 255 255) )

(gimp-file-save RUN-NONINTERACTIVE image drawable filenameout filenameout)

(gimp-image-delete image)))

Now I just invoke

: gimp -i -b '(simple-white-to-transparent foo.eps foo.png )' -b '(gimp-quit 0)'

and voilà! Actually, I've written a script called eps2pngTranspa... to do this for any specified files. n

How do I change the white argument (255 255 255) to let GIMP guess at the background colour???

What if I want to edit it? As of late March, Inkscape claims native PDF support. It also seems now to import EPS. But it's problematic.

So, for a Matlab plot, I found saving as EPS rather than SVG after edits got an uncorrupted result. Then gimp, above could do the rest...

4.1.5 Savefig

Author claims, July 2008, that it's been updated...

4.2 Adobe Illustrator 8.01

As of October 2007, Ubuntu Gutsy's wine installed Illustrator 8 with no problem. I just launched “wine setup.exe” from the Illustrator folder on the CD. Wowee. I used to use extensively (and love) this program. I still find nothing close (scribus, inkscape) for editing eps. (so am gradually switching over to svg?)

Illustrator seems to work fine for me under Wine in Windows 2000 mode, but not under Windows NT 4.0 mode, even though that's when my copy of Illustrator dates from.

April 2008: I tried the same under Hardy, with similar results. Actually, in both cases, it's really unstable, even though it loads up nicely and starts working.

Oh! How I long for an eps->svg converter.

4.3 Stata

Stata is not a favourite software of mine. But I use it heavily, though mostly through a python interface. :)

Aug 2008: Stata 10 on gnu/linux (Ubuntu Hardy) has really messed up fonts (non-proportional are proportional; keywords etc overwrite other text in output window and help window) and I cannot figure out how to fix them.

It's also absurdly slow to display coefficient tables in the output window. Other material is fast to come up, but coefficient tables come up gradually, line by line. As this is happening X hogs the processor, at least under compiz. If I switch to another desktop while it's working, the output is displayed very rapidly.

Ah, switching to Courier 10-pitch at 10pt seems to fix both problems.

But my preferences needed to be reset each time I run Stata! Why? my ~/.stata10 folder was owned by root. Oops! (Unsurprisingly result of installation process). Delete it. Now preferences are persistent.

4.4 Citrix ICA client for NFUSE under Ubuntu Hardy 8.04

I'm running Ubuntu for 64 bit but just downloaded the .tgz for version 10.6 of the ICA client.

Note that you must move the .tgz into a new directory before unpacking... some of its contents go at the top (local) level.

Run ./setupwfc

I installed as root (which puts things in /usr/lib) and chose to integrate with KDE/Gnome –> result was that Firefox did NOT know what to do with an .ica file.

I installed as a user (which puts things in ~) and chose NOT to integrate with KDE/Gnome –> now when I browsed a .ica file Firefox gave me two choices of application to deal with it. I chose the first (not default!) and got the foreign application popping up in a window: great.

Incidentally, the Citrix support people by phone were as obnoxious as any firm can choose to be. I suppose corporations can put thoroughly obtuse and unempowered people on the phone and rely on customers' frustrations to have them try Google search, which will leverage all the work of volunteers doing the corporation's support work for free. You'd think at least Citrix could synthesize some of what is on the web already.

4.5 Stata

4.5.1 Converting to Stata format from .sav (SPSS)

How to convert from SPSS (.sav) format to Stata, when all you have is Stata and free stuff? Well, Stat/Transfer is only 59$ for students, and available for unix. But there is no-cost (not OSS) software available to convert:

http://am.air.org: It's only for Windows, so I run it under VirtualBox.

It reads in SPSS and exports to Stata/SE, but the result eventually seemed inferior to Stat/Transfer's output. So I bought a license for that.

5. Ubuntu 9.10

October 2009.

This was the least troublesome upgrade in a while... but it still did not work. It left my machine without any access to the speaker/sound hardware. Couldn't figure it out.

Installing 9.10 from scratch worked. Every single release I end up giving up on the upgrade and doing a fresh install. It's not much work, since I keep two system partitions, and the commercial software just needs to be relinked from a separate, persistent /home partition.

Other tweaks that took me a while to figure out:

my xrandr scripts stopped working. I needed to replace all “LVDS” with “LVDS1” and “VGA” with “VGA1” ... The software produces no warnings with the wrong (old) names, and it is even partly but not fully functional. Weird/annoying.

6. Ubuntu 9.04

6.1 Beta upgrade

Beta Upgrade completely killed my account. Login left a white screen, stuck.

Anyway, BE CAREFUL NOT TO RUN FETCHMAIL BEFORE POSTFIX IS RUNNING. I THINK I LOST A WHOLE BUNCH OF EMAIL AS A RESULT OF THIS.

sudo apt-get install latex-beamer emacs latex2html elinks xpdf-utils ssh openssh-server python-mode emacs-extra emacs-goodies-el preview-latex a2ps psutils gv sshfs lyx latex-xft-fonts fetchmail postfix ipython python-mode alpine ubuntu-restricted-extras meld pdftk feh enscript mplayer imagemagick libdvdcss2 sbackup python-matplotlib texlive-extra-utils traceroute htop gprename pyrenamer sound-juicer

Need comment.sty

6.2 Fresh install

(later, in June).

Remember: Turn fetchmail off in .startupX before upgrading
Go through medibuntu instructions, below. Then install stuff, above.
My wired connections setups are not in new install (ie weren't in my account)...
Sound does not work in the headphone jack (the speaker works, until one plugs in headphones) after install.. Bug report: https://bugs.launchpad.net/ubuntu/+source/linux/+bug/398318 $\rightarrow$ Four months later: I found an unintuitive but simple solution, noted in that bug report.
Compiz no longer works!! The appearance dialogue says “visual effects could not be started... Unbelievably, the intel graphics card(s) is STILL blacklisted!! So one needs to do something like

: mkdir -p ~/.config/compiz/ && echo SKIP_CHECKS=yes » ~/.config/compiz/compiz-manager
to enable it.

The “Add/Remove programs” option in my menu disappeared.

More Comments:

beautifu linstaller
didn't find accounts to import, despite the partitioner seeing both existing linux partitions.
reboot in new 9.04 fails (every time): hangs, needs ctrl-alt-del at very end (followed by hard power-off)! ugh.
the printer interface is broken. It no longer sees my printer (which for several versions has been no problem / auto-recognised, etc) and does not offfer me the option to add a new printer. It turns out that “cups” was not installed, even though it didn't come up in “Add/remove” search on printing, and the service is listed in services. Installing “cups” fixed it.
had to make choices, as usual, for postfix to install... but it seems fetchmail etc all worked fine then.

Also add:

: mailutils simple-ccsm python-tk python-mechanize gnumeric mail-notification network-manager-pptp links gftp skype

There is still a bug in fetchmail. My / partition filled up (need to copy over sbackup conf when upgrade!) and as a result I LOST a whole bunch of email: fetchmail/postfix still just kills the mail when popping to a full disk. Yikes! Hard to believe such a bug still exists. I guess I've filed it in the past: https://bugs.launchpad.net/ubuntu/+source/fetchmail/+bug/138168

7. Ubuntu Hardy

First, you still need to start with adding a repository, as described at https://help.ubuntu.com/community/Medibuntu and accomplished with the following two lines.

: sudo wget --output-document=/etc/apt/sources.list.d/medibuntu.list http://www.medibuntu.org/sources.list.d/$(lsb_release -cs).list && sudo apt-get --quiet update && sudo apt-get --yes --quiet --allow-unauthenticated install medibuntu-keyring && sudo apt-get --quiet update
sudo apt-get install app-install-data-medibuntu apport-hooks-medibuntu

Then, some of my favourites:

: sudo apt-get install latex-beamer emacs21 latex2html elinks streamripper xpdf-utils ssh openssh-server emacs-extra emacs-goodies-el preview-latex a2ps psutils gv sshfs xmms lyx latex-xft-fonts fetchmail postfix ipython python-mode alpine xmms-wma xmms-mp4 ubuntu-restricted-extras wine meld pdftk feh enscript mplayer alien efax-gtk imagemagick libdvdcss2 sbackup gnumeric python-matplotlib texlive-extra-utils traceroute

Next, to be able to use an extended desktop, I unfortunately needed to edit xorg.conf. Add the line

: Virtual 2560 2560

under Section Screen, SubSection Display.

Also, I keep my mail spool on the /home partition:

: rmdir /var/mail
ln -s /home/mail /var/mail

I also have to copy over my /etc/sbackup.conf from my old installation, if any. This runs all daily backups to a remote server, and using the sbackup config GUI, “test” the remote ssh connection to verify identity.

Some more installed things:

: r-base screem virtualbox-ose virtualbox-ose-modules-generic

8. Ubuntu Gutsy

After install, use “Software sources” GUI from menu to select the sources, turning on some extra ones. Then..
Some of my standard friends:

: sudo apt-get install latex-beamer gbib aterm emacs21 latex2html elinks streamripper xpdf-utils ssh emacs-extra emacs-goodies-el preview-latex a2ps psutils gv sshfs xmms lyx latex-xft-fonts fetchmail postfix ipython python-mode alpine xmms-wma xmms-mp4 ubuntu-restricted-extras wine meld pdftk feh enscript mplayer alien efax-gtk imagemagick w32codecs imagemagick ps2edit

Which gives the response (I ignored this?):

: you must run apt-get install 10 emacs-extra/select-modes doesn't exist to install theses languages support long description here...

It also said (I ignored this?):

: msttcorefonts uses defoma Msttcorefonts uses the DEbian FOnt MAnager (defoma). If you wish to use the fonts provided by this package under the X Window System, you must configure it to use defoma fonts. The easiest way to do so is to use the x-ttcidfont-conf package. For more information, install the x-ttcidfont-conf package and consult its documentation under /usr/share/doc/x-ttcidfont-conf. For uses of msttcorefonts not related to the X Window System (e.g. printing) this is not required.

I chose “internet with smarthost” for the mail server options. I set it up to use smtp.gmail.com as a smtp relay host. (?). I also told it the local default host to use was interhcange. dpkg-reconfigure postifx internet with smarthose then cpbl-laptop chose NOT synchronous updates
I also needed to edit /etc/fuse.conf, as described in my sshfs section

That's about it! wow.

Oh, I also clicked on date/time and set it to use NTP

I also installed Adobe Acroread – see ubuntuguide.org

And I had to set up my printer – nearly automatic.

I downloaded the RealPlayer rpm and installed it with alien, as before.

8.0.1 Jan 2008

Once again, I have permanently lost email due to bad permissions in my /var/mail path. I had symbolically linked the latter to another path, and that path had not quite the right ownership, although

drwxrwsr-x 2 root mail 4.0K 2008-01-24 21:51 mail/

rather than

drwxrwsr-x 2 root root 4.0K 2008-01-24 22:13 mail/

. The other directories/files had the correct permissions.

Fetchmail got the mail. Postfix caught it, queued it, and then either bounced it or destroyed it. (Okay, maybe it wasn't permanently lost – maybe bounced.)

9. Dreamhost

visit /home/user/logs/ to see log files, including apache logs.

Endless games with .htaccess. :

The following only allows people from a certain subnet to log in, but requires a password from them:

AuthType Basic

AuthUserFile /path-to-.htpasswd AuthName Security temporarily enhanced...

Order deny,allow

Allow from 1.somesubnet.

Deny from all

require user afriend friendtwo

And with this small change, it allows log in without password for people from the subnet, but requires a password from everyone else:

AuthType Basic

AuthUserFile /path-to-.htpasswd AuthName Security temporarily enhanced...

Order deny,allow

Allow from 1.somesubnet.

Deny from none

require user afriend friendtwo

For testing. note: http://user:pw@host/dir format for links.

oh, blast. here, finally, i found the old, old version:

AuthType Basic

AuthUserFile /homegrad/path-to.htpasswd

AuthName Green College Members

require user afriend friendtwo

Satisfy Any

order deny,allow

deny from all

allow from my.ip.addres.

require user afriend friendtwo

</Limit>

STatus: in directory/testing, making a routine to determine a user access level for all other php scripts, so that differnet stuff can be served to different kind of users or form different places. continue from wireless.

Actually, the above old version works fine.

10. Server upgrades

From Ubuntu 7.04, I still can't use do-release-upgrade, despite Ubuntu's instructions. The latter command claims I need to install a desktop like gnome-desktop for it to work. !!?! But it's for servers!

Oops. The boot partition filled up! Make sure to clean up those old kernels.

11. Stata 9 for Linux/unix: how the network license manager works

Stata's tech support wrote to me to explain how things work when my license got stuck. This was very useful info that doesn't seem to be available elsewhere (?):

———–

Dear Christopher,

This could be a problem with license tracking. I think it will be helpful for me to lay out in the open how our license checking works. Also, I think I know why your user is getting 'stuck' in your license, and I have a couple of suggestions for ways to prevent and work around this.

Stata uses a simple file to track simultaneous license usage. This file is

/usr/local/stata9/.license/stata.sim

(the initial part of the path may of course vary depending on where you installed Stata).

This file contains slots for as many users as your license supports. Each slot holds four pieces of information

userid hostname instances time/date stamp

Each unique userid/hostname combination counts as 1 user. That is, if user 'bob' on host 'alpha' starts Stata 3 times, this only counts as 1 user. The license-tracking file does hold the number of instances in it, so it knows that 'bob' is running 3 Statas, but it only counts these three instances as 1 license position since all three instances are the same userid/hostname combination. When 'bob' exits one of those instances, the instance counter will go down to 2. When he exits all instances, the license counter will go to 0 and that license slot will be open for another user.

If 'bob' starts Stata once from host 'alpha' and ten times from host 'beta', this counts as 2 users (bob/alpha and bob/beta). bob/alpha will show 1 instance, bob/beta will show 10 instances, and everything works as I described in the paragraph above when 'bob' exits Stata.

While Stata is running for a particular user/host, it periodically checks the license file to make sure that it hasn't been manipulated in an attempt to get around the restriction on the number of users allowed by the license. When Stata does this, it updates the time/date stamp for the given userid/hostname combination. Think of this as Stata telling the license file yes, I am still running.

When a new userid/hostname combination starts Stata, it looks for an open slot in the license-tracking file. A open slot is defined as one where the instance counter is 0, OR a slot where the time/date stamp is 12 or more hours old. If the time/date stamp hasn't been updated for 12 or more hours, then the instance or instances of Stata that were occupying that license position are considered to no longer be alive, and so the slot is freed for use by a new instance of Stata.

Therefore, at your site, even if you have a license position which is not freed when you think it should be, it should work itself out after 12 hours assuming that there really is not a copy of Stata running and associated with that slot in the license-tracking file. The only way the time/date stamp will be updated for that slot is if there is a running copy of Stata that is updating it. If the Stata is not running, the time/date stamp will not be updated, and 12 hours later, another Stata will be able to take over that slot.

The most common reason that license positions do not get returned properly to the license-tracking file is that users

1) start a console version of Stata

2) kill that console version of Stata by closing the window in which it was running without first typing -exit- to get out of Stata

This prevents Stata from running through its normal exit code and since it has been killed, it has no chance to remove itself from the license-tracking file.

The only other ways an instance of Stata can become stuck in the license-tracking file are if Stata crashes or if the user or system administrator explicitly uses Unix's 'kill' to kill a running Stata. Both of these are extremely rare of course.

My best guess is that you have users who are using the console version of Stata in a plain text window (such as an xterm or some sort of telnet or ssh application where they are connecting remotely) and they are closing the window in which Stata is running without first typing -exit- in Stata to allow Stata to exit normally and close out its license position. Killing Stata in this manner has other bad side effects beyond the stuck license positions, such as leftover tempfiles and unsaved preferences.

As a first step, you should try to train your users in the proper way to exit Stata whether they are running the console or GUI version of Stata:

type -exit- in Stata; don't close the window in which Stata is running or kill Stata in any way other than typing -exit-.

In the case of the X windows version, Stata creates its own window and so when you close that window, Stata types an -exit- command to itself, so everything is ok.

Hopefully this will cut down on some of the problems.

As system administrator, you can see what users are holding license positions. As root, you can always start Stata even if all license positions are full. After starting Stata, you can type

who

to see the user/host information as well as the instance count. You will see something like

. who

User Host Instances ————————————————— arr ozymandias 2

1 users

If you notice that certain users are consistently 'stuck' in the license, you should work with them to make sure they are exiting Stata properly.

As superuser, you can reset the license tracking file in case of emergency. This can cause problems, however, which I will explain below.

To reset the license tracking file, start Stata as root and type

. simulinit, replace

This will empty the license tracking file.

Here is the problem with doing that: each copy of Stata that really was running knows where it should be in that file. Remember I said above that a running Stata periodically updates the time/date stamp in the license tracking file. When it does this, it verifies that the userid/hostname combination that it knows is running Stata is the SAME as the one in the slot in which it is trying to update the time/date stamp. There are three possibilities:

1) the userid/hostname is the same (true as long as you haven't reset the license-tracking file): Stata updates the time/date stamp

2) there is no userid/hostname combination there; the slot is empty (true if you reset the license-tracking file): Stata puts itself back in the slot and updates the time/date stamp

3) there is a DIFFERENT userid/hostname combination there; someone else has taken over the slot (true if you reset the license-tracking file and another instance of Stata from someone else starts before a currently-running Stata has a chance to re-check its slot): Now you have a problem. Stata has detected that something has gone wrong with the license-tracking file or that it is being manipulated. Stata is willing to forgive this...for a short time. Stata notes to itself that it wasn't able to find itself in the license tracking file. Some time later, Stata tries again. When it does, hopefully the other copy of Stata that took over the slot is gone and Stata will be able to re-insert itself into the proper slot in the license tracking file. If it cannot do this, however, Stata tells the user that the tracking file has been manipulated. At this point the user is only allowed to save their data and exit Stata. No other commands will work.

To summarize, my recommendations are

- train users to exit Stata properly

- as root, use Stata's -who- command to find out which users are consistently 'stuck' in the file; come down on them as system administrator about their bad habits

- if you absolutely have to, as root, use 'simulinit, replace' inside Stata to reset the license tracking file. As long as no new Stata jobs start by the time all the existing Stata jobs have a chance to put themselves back in the file, everything will be ok.

12. Open source GIS software

I guess my needs for census shapes analysis are:

read shape files
calculated distances
do polygon operations
plot map (projections)
graphical selection of shapes
compatibility across unix and microsoft (since government of Canada has microsoft machines for confidential data)

So, what is out there?

http://geos.refractions.net can do polygon unions and intersections.

GRASS GIS: I've added the deb source for this and installed it... but it doesn't seem to think it's installed. sigh.
gispython.org
geoTypes ... ???
Python Cartography Library, or PCL (web-display oriented?)
geopy: this is for gathering geo info from web sites.
MapServer: oriented for web site backends, though has nice polygon set operations
aha! polygon package in Python does what I want. This is the successor to PyGPC. By Jörg Rädler - joerg@dezentral.de
map projection plotting is built into matplotlib!
FWtools looks like it includes many of the other GIS packages and includes Python or is Python. Linux and Windows

13. Zareason lightlap specs

ie Asus S62E specs:

* Intel Core^TM 2 Duo T7100, T7300, T7500, T7700 800MHz FSB Processors

* 14.1'' WXGA (1280x800) TFT Color Shine And Crystal Shine Enhanced Display

* Based On The Mobile Intel 965 Express Chipset Family

* Intel Graphics Media Accelerator GMA X3100

* Supports Intel Wireless WiFi Link 4965AGN And Intel PRO/Wireless 3945-A/B/G

* Optional DVD+CD-RW Combo Module Or Super-Multi DVD/RW Module

* Up to 3GB DDR2-533/667 SO-DIMM Dual Channel Memory

* SATA Hard Drive Support-Up To 200GB

* 4 In 1 Card Reader

Specifications:

Processor & Cache Memory Intel T7700, T7500, T7300 Processor With 4MB On-Die L2 Cache. 800MHz FSB

Intel T7100 Processor With 2 MB On-Die L2 Cache. 800MHz FSB

Operating System Microsoft Windows XP Professional Edition

Microsoft Windows XP Home Edition

Microsoft Windows XP Media Center Edition

Microsoft Windows Vista Home Basic

Microsoft Windows Vista Home Premium

Microsoft Windows Vista Business

Microsoft Windows Vista Ultimate

Chipset Intel GM965 + ICH-8M

Main Memory

Intel Turbo Memory DDR2-533/677 MHz Up To 3GB

2 x SO-DIMM Dual Channel Slots

Supports Intel Turbo Memory (Robson)

Display 14.1'' WXGA (1280x800) TFT

Color Shine And Crystal Shine Enhanced Display

Video Graphics & Memory Intel Graphics Media Accelerator GMA X3100

Hard Drive 2.5'' 9.5mm HDD; Capacity Of 40/60/80/100/120/160/200 GB And 4200/ 5400/7200 RPM-SATA Hard Drives Only

Card Reader Built-In 4 In 1 Card Reader. Supports MMC/ SD/ Memory Stick / MS-Duo

Optical Drive Optional 24X/24X/8X/24X DVD+CD-RW Combo Module

Optional 2X(2.4X)/4X/24X/24X/8X24X Super-Multi DVDRW Module

Fax/Modem/LAN/WLAN/ On board 10/100/1000 Mbps Fast Ethernet Controller

Supports Intel Wireless WiFi Link 4965AGN/AG And Intel PRO/Wireless 3945ABG

Three Antennae Support

MODEM Supported V.92 And Universal PPT. Azalia Ver 2.1 Compliant

LED Status Indicator Power On/Suspend (green )

Battery Charging/Low (orange)

Bluetooth, WLAN Accessing (green )

Email (green )

Storage Access, Caps/ Number/ Scroll (green )

Interface 1 x TV-out (S-Video)

1 x Express Card 34 / 54 Support

4 x USB 2.0 Ports

1 x 4 In 1 Card Reader

1 x VGA Port(Mini D-Sub 15-Pin)

1 x IEEE 1394 A Type Jack

1 x Headphone-Out jack(SPDIF)

1 x Microphone-In jack

1 x RJ11 Modem Jack For Phone Line

1 x RJ45 LAN Jack For LAN Insert

Keyboard 19 mm Full Size With MS-Windows Function Keys

2.0mm Travel

Audio Built-In Azalia Audio Chip, With 3D Effect & Full Duplex

Built-In Stereo Speakers And Microphone

S/W Volume Up/Down

Power Management Full Feature SMI Power Management, Stand-By, Suspend To Disk, And Suspend To RAM

ACPI 2.0 Supported

Battery Pack & Life

Li-Ion 6 Cell, 4800mAh, 64W Battery Pack

Smart Battery, Charging Time, 4hrs/2.5hrs (System On/

Off) To 90%.

AC Adapter Output : 19V DC, 3.42A, 65W

Input : 100~240V AC, 50/60Hz Universal

Dimension & Weight 334 x 252 x 22.7-35.7mm (W x D x H)

2.7kg (w/o 6 Cell Battery)

Security Pre-OS Authentication By Programmable Key Code

BIOS Booting User Password Protection

HDD User Password Protection And Security Lock

Kensington Lock Hole Provided

Warranty & E-support 2 Year Asus CBB Limited Warranty

Included Software Asus Built Or Modified For Win XP, Win Vista Power4 Gear+ Hotkey: Supports Instant Key Function Defined

With ODD Software: NERO BURNING ROM V7.5.13.0

About this document ...

Log of unix software lessons, including a Zareason lightlap laptop (based on an Asus S62E) running Ubuntu Gutsy, Hardy, Intrepid, Jaunty, Karmic, ...

This document was generated using the LaTeX2HTML translator Version 2019.2 (Released June 5, 2019)

The command line arguments were:
latex2html -show_section_numbers -split 0 linuxInstallationLessons.tex

The translation was initiated on 2021-01-18