roundabout connection from my desktop to laptop

hahaha, talk about overkill...

I'm connecting to my laptop from my desktop PC (sitting right next to each other), but it's quite a roundabout way...

- Laptop is wirelessly connecting to router #1
- Desktop is plugged into wireless router #2 (with sveasoft firmware, configured to act as a bridge to router #1)
- Both machines are connected to my company's VPN.

and yet I think that's what I have to do in order to connect from desktop to laptop, and maintain the network environment I need right now. hehe

Windows XP cmd.exe file/directory name completion

In Windows XP, to enable/customize CMD.EXE file/directory name completion, open REGEDIT and go to this key:

HKEY_CURRENT_USER\Software\Microsoft\Command Processor

Here's some info on the values that may be present, and my own settings.

Value Name	Value Type	Default	My Setting
CompletionChar	REG_DWORD	0x06 (Ctrl-F)	0x09 (TAB)
PathCompletionChar	REG_DWORD	0x04 (Ctrl-D)	0x09 (TAB)
EnableExtensions	REG_DWORD	[not sure]	0x01

IE Tab FireFox Extension is great!

The IE Tab FireFox Extension is awesome! It allows you to open the current page in a new tab, rendered by the Internet Explorer engine.

Although most sites work fine in FireFox, it's an unfortunate fact of life that some sites are designed to work with Internet Explorer only. To deal with this, I've been using the IE View FireFox Extension, which allows you to open the current page in an Internet Explorer window.

It allows you to define a list of sites that will always open in IE, which is useful if you use FireFox as your primary browser, but frequently need to use some IE-only sites.

This new IE Tab extension allows you to configure the same kind of list.. sites that will always open in an IE Tab. It even supports Ctrl-leftclick to open the current IE Tab into a new Internet Explorer window, using the IE View extension! I highly recommend both of these extensions.

Simple column name list in sybase with sqsh

At my current job, we primarily use Sybase ASE for our database servers. For most of my querying needs, I use a nice little tool called sqsh, a handy replacement for the basic Sybase 'isql'.

I hate the excessive amounts of output that sybase gives me when I do 'sp_help [table_name]', because most of the time all I'm trying to do is get a list of column names for a particular table.

MySql's 'show databases', 'show tables', 'show columns', etc. seem so much more intuitive to me.

So, a couple days ago I chained together a few tools, and created an alias + function combo that let me simply type 'show_columns [table_name]', and get nothing but a quick list of column names.

Here is what I added to my .sqshrc file to make this a permanent addition to my collection of useful commands

\func show_columns_func
\echo Column list for table: ${1}
sp_help ${1}; -m vert grep "Column_name:" \sed -e 's/Column_name: *\([^ ]\)/\1/g'
\done
\alias show_columns="\call show_columns_func "

Internet Explorer: explorer.exe versus iexplore.exe

For several months, my profile on my home desktop PC was behaving strangely. Over time, the one EXPLORER.EXE process kept growing and growing in resource usage (memory + handles + everything else, it seemed), until eventually it would crash. If I logged out once in a while, and logged back in, I got a fresh EXPLORER.EXE instance with no memory leak. This was not an ideal solution, but it worked for a while.

Well, I recently read an article in the IEBlog, which announced a beta version of the Internet Explorer Developer Toolbar. This toolbar is long overdue, and although I still prefer the Web Developer Toolbar extension for Firefox, this is a big step in the right direction! They do share some similar features, and I use both every day now.

I installed it on my work laptop just fine, but when I tried to install it on my home PC, the toolbar did not show up in IE. I logged in with my wife's profile, and she (someone who doesn't need such a toolbar) could open it just fine. But not me. So, I poked around for quite a while, using various tools such as these 3 freeware tools from Sysinternals: Filemon, Regmon, and ProcessExplorer.

I sat there and compared my laptop with my desktop, with all unnecessary processes stopped or filtered, and tried to watch the effects of toggling the toolbar on and off (On my PC, even though the toolbar would not display, it was listed in the toolbars context menu and could be toggled).

Eventually, I found a major difference. On my laptop, there was one main EXPLORER.EXE process, and then for every Internet Explorer window I had open, there was a separate IEXPLORE.EXE process. On my desktop, there was one EXPLORER.EXE process, and that was it. No matter how many IE windows I had open, it was all running inside one process.

I compared the options in my desktop and laptop network and system settings, because I remembered something in there about launching new processes (in Control Panel Folder Options View, a checkbox called 'Launch folder windows in a separate process'). That was checked on both machines, but didn't affect these web browser windows.

So I did some searching to find out how to change this, and eventually found a relevant message thread which, at the end of the thread (of course), referenced a useful solution.
Add this registry value (and the key, too, if it's not there):

Key:	HKEY_CURRENT_USER\Software\ Microsoft\WindowsCurrentVersion\ Explorer\BrowseNewProcess
Value Name:	BrowseNewProcess
Data Type:	REG_SZ (String Value)
Value Data:	Yes

Phew! Adding that key did the trick, solving both my Developer Toolbar problem and my memory leak problem.

Web pages that capture keypresses are evil!

I hate it when I encounter a web page that tries to be fancy and make its own hotkeys to navigate within the page. Well, I usually hate it anyways. I like that gmail has keyboard shortcuts, and the most important thing is that their keyboard shortcuts are simple (letter keys) and only affect the behavior of the gmail site. They don't capture keypresses that would ordinarily be handled by my browser, like Alt-F (I've depended on that to open the 'File' menu in every web browser since 1995 or so, and most desktop apps as well.

But someone at AOL/Mapquest thought it would be clever to intercept Alt-[key] events (probably not a complete list, just what I noticed):

Alt-F: redirects you to their 'Find It' page (should open browser's 'File' menu)
Alt-D: redirects you to their 'Directions' page (should put cursor focus in browser's 'Location'/'Address' bar, with current location selected)
Alt-M: redirects you to their 'Maps' page (I'm not aware of any default browser behavior for this)
Alt-H: redirects you to their main (Home) page (should open browser's 'Help' menu)

While I applaud them for thinking about keyboard navigation, I don't like their choice of keys.

Another site, Uncyclopedia, captures Alt-keys as well:

Alt-H: move cursor focus to their 'history' tab at the top
Alt-F: move cursor focus to their 'search' box in the sidebar
Alt-E: move cursor focus to their 'view source' tab at the top (should open browser's 'Edit' menu)

Now, there are certainly workarounds that I could use to allow me to continue controlling my browser with the keyboard:

1. Some browser features have multiple keyboard shortcuts; I could force myself to learn different keys.
   
2. I could try to find/create a plug-in/add-on/extension for each of my browsers, to allow me to either:
   1. reassign my browser's keyboard shortcuts to work around these web sites(see #1 above)
      
   2. prevent web pages from being able to capture these keypresses, perhaps with options to toggle this feature for the current site, etc.
3. Stop using web sites that steal my favorite keyboard shortcuts
   
4. Use a different browser, such as Opera (which seems to be immune to the evil put forth by the 2 sites above)

But I just feel like I shouldn't have to do extra work because of these sites... their web developers should respect the common keyboard shortcuts used by the big players in the browser market today (IE and Firefox).

IE assumes .CSV is HTML, Firefox and Opera get it right. [part 2]

[...continued]

So, I decided to try to write a 'bookmarklet' (or 'favelet' as they're sometimes called in the IE world), because I've been finding a lot of useful ones lately, and I often find myself throwing javascript into my address bar by hand to modify a particular page's behavior. Before writing one myself, I searched for a while to see if it's been done already. I wasn't able to find any javascript code to transform a page of .CSV data into a more readable form (namely, an HTML table). I did find a CSV Converter page, which lets you paste CSV data into a textarea (or upload a file) and submit it to a CGI script, which returns the data as either HTML tables or WikiMedia format. I first tried writing a bookmarklet to submit the current page's body to that CGI script.

But that didn't work so well. Part of the problem, from what I could see, is that when IE rendered the page as HTML, the carriage-return/line-feed characters were replaced by a space. This presented a challenge for programatically parsing the CSV text, because now I had to know how many columns existed in the data, so I could figure out where to break the lines, so the table would have more than one row. (My earliest attempts produced a table with only one row, with LOTS of columns)

So, I ended up cheating a little bit, and requiring that the data in the first column on the second row will always be 'IE'. Other than that, the bookmarklet I created can handle more columnns in the future, automatically.

Right click this link and add to Favorites/Bookmarks

If you'd just like to look at the code, click here to see it in a more readable form.

Well, wouldn't you know it... this bookmarklet works in IE 5.01, IE 5.5, Firefox 1.07, and Opera 8.5, but NOT IN IE 6.0. The exact same browser whose shortcomings I was trying to work around. (The earlier IE versions also made the CSV == HTML assumption, but at least my bookmarklet worked in those versions). IE 6.0 did not generate any javascript errors when running the script, nor did it pop up its little security warning bar, or any other visible behavior. It just sat there.

I give up. Stupid IE.

To see the behavior (or lack thereof) of the bookmarklets above:

1. right click it and add it to your favorites/bookmarks


2. go to one of the CSV pages (all 3 files are the same--they just have different extensions):
   
   
   
   • MultiSelectPerformance.csv
     
   • MultiSelectPerformance.csv.txt
     
   • MultiSelectPerformance.csv.html
     
   
   


3. select the bookmarklet from your favorites/bookmarks list to run the script
   

Doing this with different browsers will unfortunately yield different behavior. If anyone can suggest how to make it work in IE 6.0, or how to fix it so it doesn't require 'IE' to be the first value on the second line of the CSV file, I'd love to hear about it!

IE assumes .CSV is HTML, Firefox and Opera get it right.

Sooo... I posted that .csv (comma-separated values) file containing some benchmarks as part of my previous post on the performance differences between browsers when populating multiple select boxes in 2 different ways. What I didn't notice until later, though, is that that file looks like crap if you click the link to it in Internet Explorer. IE seems to make an assumption like "Hmm, it doesn't have extension '.txt', so it must be HTML!" and it renders it as such. Of course, this .csv file doesn't contain any HTML tags, so it doesn't look pretty when IE makes that assumption:
MultiSelectPerformance.csv

For comparison, I made 2 copies of the CSV file:
MultiSelectPerformance.csv.txt
MultiSelectPerformance.csv.html

When I tried clicking the original .csv file link in Firefox and in Opera 8.5, they both behaved as I believe they should. They asked me if I wanted to open the .csv file with the application that is registered in Windows to handle files with that extension, Microsoft Excel, or if I wanted to save it to disk, or cancel.

Screenshots of those dialogs:

It strikes me as odd that IE, with its litigiously tight integration with Windows, didn't bother to check the Windows File Types list before jumping to the strange conclusion that .CSV is a typo for .HTML. It's possible that I could change my web server config files so it would send an appropriate 'Content-type:' header when .CSV files are served. But screw that, this is a stupid behavior in one browser, and I may want to serve a .CSV file on a web server someday where I can't touch the config files. I was determined to do something to handle it on the client side.

[to be continued...]

Comparison of Multiple Select Box Population Performance

I was recently working on developing a web application for a team project at school, when I ran into a little problem. We had a list of 1881 items to put in a select box. That, in itself, was a no-brainer. The problem was, we wanted to have 10 select boxes on this page, each with all 1881 items in it.

So, since I was using JSP, I pulled the entire list of OPTION tags out and stuck it in a separate file, then used the include directive, like:

< %@ include file="optionList.jsp" % >

inside each of the 10 sets of SELECT tags, so the list was only stored in one place. However, this optionList.jsp file was 59KB, and this page needed to load reasonably fast, despite being served off my home server with a cable modem, and viewed at school with a flaky, unreliable wireless connection. Therefore, a page with over 590KB worth of text/html was just not going to work.

Well, I've dealt with manipulating select boxes using JavaScript before, so I decided to try putting all the options into an array of strings, and looping through each select box, and each element of the option array, to add all the options. Well, this worked fine in Firefox, but in IE, it took a ridiculous amount of time (roughly 2 minutes). With that kind of performance, we might as well just load the 590+ KB original page!

So, I did some research into javascript optimization, and explored the W3C DOM spec extensively. I created about 10 different methods of populating the select boxes, and ran each of them with timing code around them, in IE and Firefox. Eventually, I found a solution that performed reasonably well in IE and Firefox, and used it for the team project.

The key lines of code are:

var tmp = document.getElementById('selectBox_0').cloneNode(true);
// ...
var selectN = document.getElementById('selectBox_' + pickNum);
// ...
selectN.parentNode.replaceChild(tmp, selectN);

Basically, I created the first select box one option at a time, then cloned that select box object (with the 'true' parameter, to clone all child nodes as well) and replaced all the other select boxes with the clones. Then, I set some properties on the clones to make them unique (IDs, default blank options, etc). Dramatically faster than the other.

I've put together a benchmark page to test/demonstrate the original method of populating the select boxes (adding one option at a time, to one select box at a time), versus the method I found that worked so well. Then I ran the benchmark on several browsers, on several computers, and had some friends and co-workers try it, too. You can find the benchmark page here:

http://www.oatmealcookies.org/blogdl/
performance/MultiSelectPerformance.html

The benchmark results, as I've recorded them so far, can be found in a comma-delimited text file at:

http://www.oatmealcookies.org/blogdl/
performance/MultiSelectPerformance.csv

I recently found out that Opera 8.02 works about as fast as Firefox, even faster in some cases. I was impressed that the code that I'd tested thoroughly in IE and Firefox (but not Opera) actually RAN without errors, but the fact that it ran so fast was pretty cool.

Please feel free to run the benchmark and send me your results! The way that these measurements have been recorded is:

Click 'Run All'.. when tests are complete (ranges from 12 seconds to 5 minutes, depending on browser and platform), click 'Save Data', which populates a textarea at the bottom of the page, and selects it, so you can then copy it to your clipboard and post it in a comment on this blog entry. Please include all fields that are in that CSV file, including browser versions, OS, processor speed, etc.

Google search results that require subscriber-only login? Disable JavaScript!

Today I experienced something I've seen many times before, but it pissed me off enough to investigate this time. Many times, when I search for some info with google, some of the results that I click do not include the matched content, but instead take me to a login page, saying the content is restricted to subscribers only.

Today, the site was techtarget.com. I've found useful information on that site before, and it was the #3 result in my search (I searched for "tracking down wireless network interference"). Sometimes, it's possible to view these pages by viewing google's cached copy. This time, that didn't work either. Even the cached copy redirected me to a login page.

So, i tried looking at the http headers with LiveHttpHeaders extension in FireFox, and that's where I saw these URLs as the first two requests:

First, google's cache:
http://64.233.187.104/search?q=cache:JGSD_2iKWo0J:
searchmobilecomputing.techtarget.com/tip/1,289483,
sid40_gci1109438,00.html+tracking+down+wireless+
network+interference&hl=en&client=firefox-a

Second, a javascript page from techtarget:
http://searchmobilecomputing.techtarget.com/
pageScripts/0,294327,sid40,00.js

Everything after that was related to the techtarget login page.

So, the redirect is in the javascript, eh? So I used 'wget' to download the javascript file ("0,294327,sid40,00.js"), then opened it in my text editor ( TextPad), and searched for 'redirect'. I think the following is causing the redirect, although I haven't looked thoroughly through the whole script, because I didn't need to:

/*
   Force login if content is members only.
   Redirect to top in case document is loaded 
   in a frame or iframe.
*/
var forMembersOnly = 
  (typeof JSmembersOnly != "undefined" && 
   JSmembersOnly == "on") ? true : false;
if (forMembersOnly && loggedIn != 1) {
  var NextURL = window.top.location.href;
  if (NextURL.indexOf(
    "/ttCMAv2/Production_Center/" +
    "Preview_Form_v2/1,2563,,00.html") == -1) {
    var redirectURL;
    if (typeof JSmembersLoginURL == "undefined" && 
        JSmembersLoginURL == "") {
      redirectURL = 
      (typeof JSlinkLogin != "undefined") ? 
      JSlinkLogin : 
      "http://searchMobileComputing.techtarget.com/" + 
      "loginMembersOnly/1,289498,sid40,00.html";
    } else {
      redirectURL = 
      ((JSmembersLoginURL.indexOf(
      "http://searchMobileComputing.techtarget.com") == 
      0) ? "" : 
      "http://searchMobileComputing.techtarget.com") + 
      JSmembersLoginURL;
    }
    window.top.location.replace(redirectURL + 
    "?NextURL=" + escape(NextURL));
  }
}

So, I just disabled javascript in my browser before clicking the link!
This is VERY EASY to do with FireFox and the
Web Developer Toolbar extension.

In my opinion, there's nothing wrong with doing this. They made the content available for google to index, but then deliberately prevented me from viewing it (bait and switch?). I just viewed it without javascript. It probably would have also worked to just use a text-only browser like links or lynx. This will probably not get around ALL cases of this annoying search result behavior, but it did help in this case.

switching between tabs (worksheets) in Microsoft Excel

It's been annoying me for a long time that Ctrl-Tab and Ctrl-Shift-Tab don't switch between worksheets in the currently open workbook in Excel.  I haven't dealt with changing it yet, but at least found what the existing keyboard shortcuts are.

 

Found this in:

Help | Microsoft Excel Help

 

then:

Contents | Accessibility in Excel | Keyboard shortcuts

 

under section: Keys for workbooks and worksheets

 

expand: Work with worksheets

 

CTRL+PAGE DOWN

Move to the next sheet in the workbook.

CTRL+PAGE UP

Move to the previous sheet in the workbook.

 

How to reattach to GNU screen sessions in windows (cygwin)

I use a Windows version of the GNU Screen window manager/terminal multiplexer every day, and wanted to share a trick that I've learned over time.  A lot of people have trouble with reattaching to existing screen sessions in the windows/cygwin version. You can probably accomplish this different ways, but this is what I have been doing every day at work for at least 9 months now:

 

1. open first bash shell window
2. run: screen
3. open another bash shell window
4. run: screen -x
   
   NOTE: this window will probably not reattach, at least it doesn't for me.  if it does, these instructions are useless for you.
   
5. close second bash window (opened in step 3).
6. open another bash shell window
7. run: screen -x

At this point, both bash windows are now connected to the same screen session.

 

Variations on this may work.. I've done this in a lot of different conditions, including of course ssh-ing into a box and attaching to the existing screen session, and also using ssh-agent just right to avoid having to type passwords all day long.  More on that later, but for now, I just wanted to get this basic information out there. 

 

The main idea is that the first attempt to attach to an existing screen session will fail, but if you try again in a new window it should work.

 

 

cvsrecent.bsh

This is a bash shell script to get a list of CVS revisions and log messages for all checkins within the past (some number) of days.

I'm using this to help me when writing weekly status reports, so I can make sure I don't forget anything.

It searches recursively, starting from the current directory, for all files that contain a '.' in their names (I only care about *.jsp, *.java, *.xml, etc. at this point), and whose paths do not include '/CVS/' or '/build/'. For each filename it finds, it checks the number of lines that cvs log returns for that file, based on the number of days back that you tell it to look, and if that number is greaterthan the number of lines returned for an unmodified file, the relevant cvs log info is displayed.

Usage: cvsrecent.bsh [DaysAgo]

[DaysAgo] is the number of days past that you want to get cvs log info for. I usually run it with '7' to get the past week's worth of cvs log messages.

NOTE: You will want to have your system configured to not require a password everytime you run the 'cvs' command, or you will get prompted for the password WAAAAY too many times. I do this using ssh-agent.

#!/bin/bash

## cvsrecent.bsh
##
## Author:
## Dave Nelson
## spugbrap.com+blog@gmail.com
##
## Source:
## spugbrap's random geek notes blog
## http://spugbrap.blogspot.com
##
## Debug version available from:
## http://www.oatmealcookies.org/blogdl/cvsrecent-debug.bsh
##
## Date Posted:
## 07/08/2005

# No [DaysAgo] parameter specified
if [ -z "$1" ]
then
  echo "Usage: `basename $0` [DaysAgo]"
  exit -1
fi

DaysAgo=$1
RefDate=$(date --date="$DaysAgo days ago" +"%Y-%m-%d")
TempFile=/tmp/cvsrecent.$PPID
JunkLogLines=11
fileList=`find . -mtime -$DaysAgo -type f -name '*.*' |            grep -v '/CVS/' | grep -v '/build/'`

for curFile in $fileList
do
  cvs -Q log -SNd ">=$RefDate" $curFile >$TempFile

  totalLogLines=`cat $TempFile | wc -l`

  if [ $totalLogLines -gt $JunkLogLines ]
  then

      echo $curFile

      tail -`expr $totalLogLines '-' $JunkLogLines '+' 1`         $TempFile
  fi
done

Offline NT Password & Registry Editor

This is an incredibly useful tool:
http://home.eunet.no/~pnordahl/ntpasswd/

It saved me when I corrupted the MBR on my work laptop's hard drive,
and didn't have the Administrator account's password (required by
default to enter the Windows XP Recovery Console).

It's a very small/minimal linux kernel, with a few powerful tools.
You can download and create a boot floppy or boot CD, and when you
boot from it, you can change your Administrator password, or even edit
your registry!

misc notes on my recent experience with parallelknoppix, clusterknoppix, and fedora

This is far from complete, and could use a lot of detail such as links
to the web sites I mention in here, etc. Maybe I'll update it later
with that information.

- can't write to NTFS in Parallel Knoppix
- openmosix terminal server lets slave nodes boot from image stored on
master HD, but that requires:
- slave notes have network boot capability (PXE)
- may require BIOS flash upgrade
- may need to change options in BIOS setup
- enable network boot/PXE option
- change boot order to try network first
- master needs to load drivers for relevant NICs on slaves
- sometimes challenging to find which driver
- pcimodules
- dmesg
- lspci -v
- creative google searching
- drivers in other OSes on that machine may provide clues
as to exactly which model number/chipset/etc. a NIC is, so you can
then google those numbers in search of info on which driver to use in
knoppix.. check hardware properties in windows device manager, etc.,
and look at driver versions/details.
- checking all drivers in list requires much more disk space on master
- GUI driver checkbox list very slow
- i liked to just edit the terminal server startup script,
modifying the regular expression that checks certain checkboxes on the
list by default. but that was probably more trouble than it was worth
to try and tell someone else to do.
- if BIOS doesn't support PXE, download driver in a boot disk
image from the etherboot project, at http://rom-o-matic.net, then
write that image to a floppy with RAWRITE tool, and boot a slave from
that floppy.
- need to copy cd image to master HD for slaves to remote-mount
- plug master and slaves into one hub/switch, isolated from internet,
etc, and disable any network hardware that is not relevant. this may
not be required, but it simplifies things, so it's more likely to
work.
- couldn't get parallelknoppix to use my USB HD with FAT partition for
permanent storage for the terminal server. could use the drive
normally in parallelknoppix, but not for the main purpose i needed
non-NTFS storage for.
- machines with too little memory couldn't have a large enough ram
disk, so they didn't want to be master

- tried in vain to get clusterknoppix or parallelknoppix working on my
network at home, attempting with 6 different machines, in various
combinations, spending a total of probably 40 hours on this task. it
pains me to say that I never did get a useful cluster working at home.
luckily, our group was able to get one working between our 4 laptops,
in a matter of only about 4 hours, including recording a 30 minute
video of the process after somewhat perfecting it.

- when a master's terminal server is running and slaves are connected,
they mount a directory on the master, and they are able to read/write
files anywhere in that directory tree. supposedly this is not
designed to be secure, it's designed to be quick and easy and used in
environments that are as secure as they need to be.

- ClusterKnoppix
- hardware support nicer for me (such as mouse buttons)
- includes CaptiveNTFS for mounting NTFS partitions read/write
instead of read-only
- openmosix viewer seemed to automatically see other
ClusterKnoppix machines on the LAN, and automatically clustered them
and showed their processor usage in the one viewer window, but that
only worked for me two weeks ago. last week when I tried again, I
couldn't get any 2 machines in my house to recognize each other as
nodes to cluster.

- challenges I encountered with Fedora:
- tried to install fedora on several machines, but the only
install that really worked out well was on a machine that I dedicated
to fedora.
- I let fedora start with an empty hard disk and partition it
automatically, etc. that machine worked out fine, and that's what I
ended up doing my individual assignment on.
- was able to easily:
- use sendmail for local email
- create samba shares that my windows machines could mount
- read ntfs, and write ntfs as well after i played
with the mount command/options
- set up web server, vnc server, etc
- one machine took 7 hours to install from the 4 fedora CDs.
painful. then, it was not even able to start up after the install.
kept hanging during the fedora startup sequence.
- another machine had the same kind of hanging issue, but did
not take nearly as long to install initially.
- tried obsessively to get fedora to install and boot off of
an external USB hard drive.
- various discussion threads can be found by googling
which explain step-by-step how people have accomplished this.
- it was not simple/straightforward whatsoever, and I
never did get it to work, after spending probably 20+ hours on it over
the course of several days.
- getting it to install on the usb drive was the easy
part. getting it to boot from it was not.
- the main reason for trying to do this was because most
of my machines only have NTFS partitions, and fedora wasn't crazy
about that fact. so I wanted to be able to install it on FAT or ext3
partitions on the external HD, so I could avoid messing with my work
laptop's hard drive and stuff.
- managed to corrupt the MBR on my work laptop in the process
- found useful NT admin password resetter tool, which
allowed me to then boot from the windows XP setup CD and go into the
Recovery Console, where the FIXMBR command saved me.
- two other machines at home were disqualified by the fact
that they only had NTFS partitions.

miscellaneous search/replace regular expressions I've used recently in TextPad

Here are some miscellaneous search/replace regular expressions I've used recently, that seemed worth keeping (to me). I used these in TextPad with the option in Configure Preferences Editor, called "Use POSIX regular expression syntax", checked. Sometimes I find that TextPad's search/replace functionality is more useful to for a particular purpose than 'sed' in unix/cygwin. Other times I use sed and stuff. That's the beauty if cygwin, I can mix and match windows and unix whenever I want (as some previous and near future posts will show).

These are mostly presented in the format:
Line 1: short description of search/replace expressions
Line 2: regex to search for
Line 3: replacement expression

change from (myMaybeNullFunc().equals("myStaticString") to ("myStaticString".equals(myMaybeNullFunc())
([\(]*)([a-zA-Z\.\(\)]+).equals\("([^"]+)"\)
\1"\3".equals(\2)

change from (myMaybeNullVar.equals("myStaticString") to ("myStaticString".equals(myMaybeNullVar)
([^a-zA-Z])([a-zA-Z]+).equals\("([^"]+)"\)
\1"\3".equals(\2)

change from myResultSet.getString("foo") to myObject.getFoo()
myResultSet\.get[A-Z][a-z]+\("([^_"])([^_"]+)"\)
myObject.get\u\1\2()

change from myResultSet.getString("foo_bar") to myResultSet.getString("FooBar")
rs\.get([A-Z][a-z]+)\("([^_"])([^_"]*)_([^_"])
rs.get\1 ("\u\2\3\u\4
(run this one repeatedly until no more matches, then run the one above it)

change HTML start tags to upper case
<(\<[[:word:]]*\>)
<\U\1

change HTML end tags to upper case
</(\<[[:word:]]*\>)
</\U\1

TextPad 'tool' definition to open generated .java file based on currently open .jsp file

Set this up as a Tool in TextPad, and dropped the sed script (at the bottom of this post) into my cygwin home directory's bin directory (~/bin/), and now I easily open the .java file that tomcat generated when it last compiled the .jsp that I'm currently editing. This is helpful for debugging sometimes, when the compilation error message does not include the .jsp file line number, just the .java file line number.

Command:
c:\cygwin\bin\bash.exe

Parameters:
-c "'/cygdrive/c/Program Files/TextPad 4/textpad' \
`echo '$File' sed -f ~/bin/GeneratedJava.sed`"

Initial Folder:
c:\

Checked options:
Run minimized
Close DOS window on exit

~/bin/GeneratedJava.sed contents:
#!/usr/bin/sed -nf

s/path\\to\\my\\jsps/apache\\tomcat5\
\\work\\Catalina\\localhost\\mywebappname\
\\org\\apache\\jsp/

s/\.jsp/_jsp.java/

sorted list of number of changed lines in files that need to be checked into cvs

Sometimes I make one change to a set of files, sometimes even a large set of files. Say, for instance, that I changed which java package a class resides in. Then, I'd probably run a set of commands that would search and replace the import statements in all files that use that class. But, at that point I might now have 20 files in my source tree that need to be checked in, which all have the exact same change, to just one line in each file.

If my local source tree was up to date with CVS before that, the checkin would be easy. I could just cvs commit at the root of the tree, and enter one comment ("changed package of class X from a.b.c to a.d") to describe the change to all of them.

But, usually I've got at least a couple other things that need to be checked in, so I have to either go through all of the files, one at a time, or do something automated. The following is one set of commands that has proven useful to me for this purpose. I've got another, but it's not quite ready to share (that will come soon).

The commandline below will safely and quietly cvs update to get a list of modified files (marked with an "M"), then run cvs diff on each file and count the lines in the diff output.. then they are all sorted numerically by number of changed lines, and displayed. I can easily see that all of the files with 5? (i forget exactly.. however many lines show up in a 'cvs diff' for a single changed line in a source file) changed lines, and can probably check them all in with one comment.

(for f in `cvs -q -n update grep '^M' sed -e 's/^M //g'`; do echo `(cvs diff $f wc -l)` $f; done) sort -n

Sybase ASE 12.5 error message about 'select into' option needing to be enabled

Sybase (ASE 12.5) error message I was getting when trying to modify a column's metadata/attributes:
   The 'select into' database option is not enabled for database 'myDbName'. ALTER TABLE with data copy cannot be done.
   Set the 'select into' database option and re-run.

 

Solution that worked for me (including the column-altering command in the middle):

USE master;
EXEC sp_dboption "myDbName","select into/bulkcopy",true;
COMMIT;
USE myDbName;
ALTER TABLE myTableName
MODIFY myColumnName varchar(60) NOT NULL;
USE master;
EXEC sp_dboption "myDbName","select into/bulkcopy",false;
COMMIT;

References:

Ed Barlow's "fix_db.pl" script, which I happened to find the solution in when grepping my local machine.
http://www.edbarlow.com/document/utilities/readme.htm

This page, where I stumbled across the solution eventually, as one of many google search results.  I had result pages open in several windows, and was looking through them when I noticed the grep results.
http://www.devx.com/vb2themax/Tip/18583
 

Build with ant, open source files in TextPad, jump to lines with compilation errors

I use cygwin and textpad in my day-to-day work, among many other things. To compile my current java project, I use ant. I'm always interested in integrating utilities to make everything more convenient, so this afternoon I set out to create an entry in the textpad "Tools" menu, to run ant and compile my source code.

I found what seemed like useful information here, so I was able to at least run ant from textpad. However, compilation errors would show up in the "Command Results" window, and the part on that web page about being able to click on a compilation error to jump to the offending file and line number was not working for me. I tried messing with the regex and switching my textpad regex type from POSIX to the textpad default type, but I could not get it to work.

So, I went a different route. I decided to attack it from the bash shell angle, by running ant and manipulating the output. What I ended up with is one string of commands that runs ant, then opens all source files with compilation errors into textpad, and jumps to the line numbers that the errors occurred on:

ant 2>/dev/null ¦ grep "\[javac\] [A-Za-z]:" ¦ sed -e "s/^ *\[javac\] \([A-Za-z]:[^:]*\):\([0-9]*\):.*$/\1(\2)/g" > /cygdrive/c/temp/textpadant.txt ; "/cygdrive/c/Program Files/TextPad 4/TextPad.exe" @c:\\temp\\textpadant.txt &

I will probably try to expand on this a little further when I have time, so I can initiate all this from inside textpad, as an entry in the "Tools" menu. It should be pretty easy, but I don't have time to mess with it any more right now. For now, the solution I came up with will already save me some time every day.

-dave

imposing some of my programming style on a whole source tree of files (muahahaha!)

I inherited a project at my current job that had only been worked on by one person before, over a long period of time. I've been refactoring it for quite a while, now, and one thing that I've done a lot of during this is reformat into a style that I'm used to/more comfortable with. I've done this largely with the help of various sets of regular expressions, using either search and replace in TextPad (my text editor of choice) or a combination of find/grep/sed/etc.

Here's one that I did fairly recently, which, while not incredibly useful or complicated, did what I needed it to do. I also had fun figuring out the right expressions, and enforcing my programming style on hundreds of files with one commandline. :)

for f in `find . -name '*.jsp'`; do echo $f; sed -e 's/^\([ \t]\+\)\(.*[^ \t]\+.*\)[ \t]*{$/\1\2\n\1{/g' -e 's/^\([ \t]*\)}[ \t]*\([^ \t]\+.*\)$/\1\}\n\1\2/g' $f >$f.2; diff $f $f.2; mv $f.2 $f; chmod 755 $f; done

I created this in two parts, originally in textpad, then decided I wanted to put them together and run them on all of the JSPs in the source tree. Here are the two parts, as I documented them originally, which should make it easier to understand the final command above:

----
move {'s from ends of code lines to blank lines,
for example:
    try {
would change to:
    try
    {

search for:
^([ \t]*)(.*[^ \t]+.*)\{\n
replace with:
\1\2\n\1{\n

sed -e 's/^\([ \t]\+\)\(.*[^ \t]\+.*\)[ \t]*{$/\1\2\n\1{/g' FileName.jsp
----
move }'s from beginnings of code lines to blank lines,
for example:
    } catch (Exception e)
would change to:
    }
    catch (Exception e)

search for:
^([ \t]*)\}[ \t]*([^ \t]+.*)\n
replace with:
\1\}\n\1\2\n

sed -e 's/^\([ \t]*\)}[ \t]*\([^ \t]\+.*\)$/\1\}\n\1\2/g' FileName.jsp
----

-spugbrap

remote access to a firewalled machine with ssh port forwarding

At some point last year, I needed to be able to remotely control a computer at work that could not be accessed from the outside world due to a firewall. I already knew the power of ssh port forwarding, and figured there was some way to accomplish what I wanted to do using ssh.

Sure enough, I was able to put together the following two commands, which allowed me to do what I needed to do, with the help of my computer at home (whose firewall I could actually open a port on). I only did it once, just to see if it could really be done, but I did not want to risk getting in trouble for making a hole in the firewall. It was a fun exercise, though, and I wanted to share it with anyone who cares to listen.

The following represent things to modify in the commandlines below, to suit your own needs:

HOME_IP = the ip address of my machine at home
HOME_USER = my username on my machine at home
RESTRICTED_HOSTNAME = the hostname of the machine I'm trying to open up
RESTRICTED_USER = my username on the machine I'm trying to open up

First, I executed this command from the machine I wanted to open up to the outside world:
ssh -g -R 8022:localhost:22 HOME_USER@HOME_IP

This forwards my local port 22 (on the restricted machine) to my home machine's port 8022
-g tells it to allow remote hosts to connect to locally forwarded ports


Next, I executed this command in that ssh session to my home machine that I created with the first command:
ssh -p 8022 -N -f -g -L 15900:RESTRICTED_HOSTNAME:5900 RESTRICTED_USER@localhost

This forwards the VNC port (5900) on the machine I'm trying to open up to port 15900 on my home machine.
-g tells it to allow remote hosts to connect to locally forwarded ports
-N says I just want to make this connection for port forwarding, not to actually execute a command
-f says to put ssh into the background after establishing the connection


Now, from any machine with an internet connection and a VNC client, I can open a VNC connection to HOME_IP:15900, and I will be remotely controlling the restricted machine.

Note that this only requires me to have ports 22 and 15900 open on my firewall at home.%

Time to start using this blog for what it's made for!

I've been continuing to keep a text file with my favorite scripts, regexes, commandlines, utilities, etc., but have totally slacked as far as posting that stuff in this blog. I had a feeling that would happen, which I said in the intial blog description post, but I want to go ahead and start putting some of that info in here now.

I was just going to post that entire text file, but I realized I need to sanitize it a bit to remove proprietary information. Plus, the file is pretty bare, and could really use at least a LITTLE documentation, particularly because some of the regular expressions are hard to follow without some idea of their purpose.

So, I will start posting bits and pieces from that file, with some description, hopefully on a regular basis. You can expect the first post later tonight or tomorrow morning. Again, these are little things that were useful to me in some way, which I wanted to save for future reference. I hope some of it can be of use to other people.

Also, if anyone knows other blogs with similar purposes (utils, scripts, code snippets, useful commands/combinations of commands, etc.), please share the URL in a comment in this blog.

-dave (spugbrap)