A piggy bank of commands, fixes, succinct reviews, some mini articles and technical opinions from a (mostly) Perl developer.

Why not to use a pop-up window

If you're using a pop-up window on your website, you're doing it wrong.
It's akin to using nested tables for display, fer chrissakes!

Reasons why it's wrong:

  • It's a poor user experience because the user doesn't get any of the usual browser functionality in the pop-up window, like the back button or the icon that tells you the page is loading.
  • I'm sure I'll think of some more later

Testing Catalyst

Notes:

Catalyst::Test - you get a context object back

Test::WWW::Mechanize::Catalyst - you don't have to run a separate server

Change timezone in Linux

Change this link:

$ ls -l /etc/localtime

lrwxrwxrwx    1 root     dwebadm        34 Dec  2 01:12 /etc/localtime -> /usr/share/zoneinfo/Asia/Singapore*

Cannot install Java JDK: semicolon found in selected path

There's no semi colon in the install path. But you still get the error:

        semicolon found in selected path

Solution is to move the install .exe to c:\ and run it again.

Possibly with command line switches: /v"/L c:\install.log"

(Search results show that this same issue has existed for 11 years!)

Why not to require javascript

Reasons not to require javascript for critical features of your website:
  • It can slow down the user experience.
  • Standard HTML form and browser controls are the way they are for a reason: They work. Subvert or reinvent them at your peril.
  • It's much more likely Javascript code won't work as expected in all browsers, than HTML.
  • When some part of it breaks, forcing the user to disable it, they won't be able to use the rest of the website
  • Not all browsers have javascript, for example Lynx (text only) and JAWS (i.e. screen readers which blind people use).
  • It often makes automated testing harder
  • When the user gets logged out due to inactivity, the AJAX calls may behave unexpectedly

Disclaimer: This is my opinion only, based on personal experience.

Select sub-section of putty buffer only

Are you sick and tired of waiting for your terminal to scroll down when dragging to select a large portion of the scrollback buffer in putty?

The putty developers already thought of that!


  1. Go to Putty Configuration window
  2. Choose "Selection" from category on the left of the window.
  3. Under 'Control use of mouse', choose 'Compromise (Middle extends, Right pastes)' if not already chosen.

(source)

Use Data::Compare to compare data outside a test

Need to compare data outside of a test, without generating "ok" / "not ok" TAP output? Use Data::Compare.

Perltidy docs / example config

Docs

Reference for all the possible perltidy options
More detailed explanation of some the options

Team perltidy options

ruleexplanationnotes
-i=4
4 column indentA lot of the legacy code has 2 column indent.
-pt=2
"horizontal tightness" 2 - no space for parenthesis tokens
 if ( ( my $len_tab = length( $tabstr ) ) > 0 ) {  # -pt=0
 if ( ( my $len_tab = length($tabstr) ) > 0 ) {    # -pt=1 (default)
 if ((my $len_tab = length($tabstr)) > 0) {        # -pt=2
(docs)
-wls='=>'
ensure there's always space to the left of =>
-wrs='=>'
...and always to the right of =>
-vtc=0
always break before a closing token (default)i.e. non-block curly braces, parentheses, and square brackets docs
-l=79
Lines should be no more than 79 characters wide
-nolc
no "outdenting" for long comments - indent them properlyBy default, full-line (block) comments longer than the value maximum-line-length will have their indentation removed ("outdented"). This rule prevents that. docs
-nolq
no "outdenting" for long quotes/stringsdocs
-sot
stack opening tokens - same as -sop -sohb -sosbPut opening parentheses, braces, etc. on the same line docs
-sct
stack closing tokensPut closing parentheses, braces, etc. on the same line docs

Options that were removed

ruleexplanationnotes
-lp
line up parenthesesSee docs
-vt=2
define opening vertical tightnessPut opening parentheses, braces, etc. on the same line. Related to -lp docs
-vt=0
always break a line after opening tokendocs
-st -se
use as a filterInstructions to the parser docs
-nsfs
'for' loop semicolon spaces
The default is to place a space before a semicolon in a for statement, like this:
    for ( @a = @$ap, $u = shift @a ; @a ; $u = $v ) {  # -sfs (default)
If you prefer no such space, like this:
    for ( @a = @$ap, $u = shift @a; @a; $u = $v ) {    # -nsfs
then use nsfs.
-nbbc
No blanks before commentsBy default, a blank line will be introduced before a full-line comment. This rule prevents that.

SSH tunnel

ssh -N -R 5555:1.2.3.4:666 user@hostname
  • 5555 is a port you make up
  • 666 is the port to which you are forwarding
  • 1.2.3.4 is the IP of the target host to which you are fowarding
  • hostname is the host where port 5555 will be made available
  • you run this command on a third machine (e.g. your local host)

This results in  hostname:5555  getting forwarded to  1.2.3.4:666

Useful bash flags for scripts verbose trace expand error exit

Useful flags:
set -v # "verbose" - echo commands
set -x # "xtrace" - like verbose but expands commands
set -e # "errexit" - abort script at first error
set -u # "nounset" - strict mode: undefined variable causes an exit

Long format:
set -o verbose

Bash "strict mode":
#!/bin/bash
set -euo pipefail
IFS=$'\n\t'

Windows: folder or file open in another program

When you try to move a folder:

    "The action can't be completed because the folder or a file in it is open in another program"

This stunningly unhelpful error message has persisted through many versions of Windows.

The way I work around it is:

  • Rename an unrelated file or folder nearby, then rename it back again.
  • The original folder or file should now be "unlocked"
You can always reboot to fix it, too.

Ctrl-C doesn't interrupt loop in Bash script

If you want Ctrl-C to be able to stop your loop in Bash, put this inside the loop:

trap "echo Exited!; exit;" SIGINT SIGTERM

(source)

SSH warning: authenticity of host can't be established

Problem

The authenticity of host 'example.com (1.2.3.4)' can't be established.
RSA key fingerprint is ab:cd:ef:12:34:56:78:90:ab:cd:ef:12:34:56:78:90
Are you sure you want to continue connecting (yes/no)?

Solution

ssh -o UserKnownHostsFile=/dev/null -o StrictHostKeyChecking=no

Make sure you understand the security implications of doing this (Hint: they're not good).

(source)

Advanced subversion usage

Rebase branch on trunk

cd /working/dir/path/to/branch

svn log --stop-on-copy | tail -4 | grep ^r | cut -d' ' -f1 | sed 's/r//'
1111

svn propget db:::dweb::trunk:last_merge_from /working/dir/path/to/branch
2222

svn merge -r 1110:2222 https://remote/path/to/trunk /working/dir/path/to/branch

# check for conflicts and test

svn commit -m'Pull from trunk'

Diff branch against trunk

svn propget db:::dweb::trunk:last_merge_from /working/dir/path/to/branch
1111

svn diff https://remote/path/to/trunk@1111 https://remote/path/to/branch

When to use "or" and when to use || (double pipe) in Perl

Use || for assigning to a variable:

$a = $b || $c

Use "or" to control program flow:

$a = $b or die 'error: $b is unexpectedly false'

Mnemonic: || is a symbol like variables and numbers are, while "or" and "die" are both english words.

(source, source)

Terminal does not wrap properly in linux

Sometimes lines don't wrap correctly in a bash terminal in linux - long lines overlap and overwrite the beginning of the line instead of continuing onto the next line.

To fix it:
shopt -s checkwinsize

To check it:
shopt  | grep checkwinsize
You should see:
checkwinsize    on

If it still isn't working, try:
reset

(source)

Create a lot of random data files on linux

# Create 10Mb random data

dd bs=1024 count=10240 < /dev/urandom > data

# Split into files of 10K each

split -a 5 -b 10240 data 'split.'

Client-side web design frameworks

A short list:

  • Foundation - doesn't work in IE8, and doesn't even degrade gracefully
  • Bootstrap - from Twitter. Works in IE8
  • Alternatives are available

See all commit messages in svn log

After merging a branch to trunk, subversion does not include commit logs from branches in the trunk log. With a little work it can be persuaded to display the log in a more useful way, see "svnlogg" perl script below.

When viewing this log output you may want to see the diff of one of the changes and naively type:

    svn diff -c 69271

...this may produce no output. However, if you type this intead:

    svn diff -c 69271 https://www.example.com/repo/branches/feature_xyz

...then you will see the expected output.

How to determine the branch name from the commit message is left as an exercise for the reader.


"svnlogg" perl script:

#!/usr/bin/env perl

# First, get all the commits.
# -v shows the filenames of changed files,
# -g shows commits from all merged branches.

my $LOG_TEMP = "/tmp/log.txt";

system("svn log -v -g > $LOG_TEMP");

# Then split the list into individual commits, filter out ones we don't want to see, and sort by date:

open(my $fh,"<",$LOG_TEMP) or die "cannot open file $LOG_TEMP";
local $/="------------------------------------------------------------------------"; # define the record separator
my @a = <$fh>; # populate an array with the commit messages
foreach my $aa (
    sort {
        # sort the commits by date (descending)
        my ($c) = $a =~ /\s\|\s(\d\d\d\d\-\d\d\-\d\d)\s/; # extract date field
        my ($d) = $b =~ /\s\|\s(\d\d\d\d\-\d\d\-\d\d)\s/; #
        $d cmp $c
    } @a
) {
    # exclude unwanted messages, especially the hundreds of "remove svn:mergeinfo"
    # also exclude the commit messages we use by convention in our organisation
    # to refer to branches and merges
    if ($aa !~ /(remove svn:mergeinfo|branching|final pull)/i) {
        print "$aa\n";
    }
}

Depending on your hardware it may be unusably slow, so pipe it into a file for viewing later.

Outlook 2007 autocomplete doesn't work

It really doesn't.

I tried these things:
  • Adding the name to my contacts list
  • Composing and sending an email to the contact (source)
  • Tools | Address Book | Tools | Options | Check names using these address lists | Contacts
Some names are auto-completed, but some aren't.

:-(

Do you know how to make it work? Leave a comment

Git: Change author name in all previous commits

On any branch, or trunk:

git filter-branch --commit-filter 'if [ "$GIT_AUTHOR_NAME" = "Josh Lee" ];
  then export GIT_AUTHOR_NAME="Hobo Bob"; export GIT_AUTHOR_EMAIL=hobo@example.com;
       export GIT_COMMITTER_NAME="Hobo Bob"; export GIT_COMMITTER_EMAIL=hobo@example.com;
fi; git commit-tree "$@"'



Linux package managers

A list:
  • rpm
  • yum - works with rpms
  • zypper (suse)
  • pkgsrc (cross-platform, builds from source)
  • yast (works on suse)

Debugging Perl in Emacs

Notes only:

  • http://search.cpan.org/~yewenbin/Emacs-PDE-0.2.16/lib/Emacs/PDE.pm
  • http://www.emacswiki.org/emacs/PerlLanguage
  • http://www.emacswiki.org/emacs/CPerlMode
  • http://cpansearch.perl.org/src/YEWENBIN/Emacs-PDE-0.2.16/lisp/doc/QuickStartEn.html

Where to download oracle wallet mkstore

It looks like mkstore was the utility in 10r2 but by 11g it had been deprecated by orapki.

An article mentioning both tools.

Linux window managers

Window managers:

  • Mate (fork of Gnome 2),
  • Mint (Extensions for Gnome 3),
  • or Cinnamon (Fork of Gnome 3)

Gnome 2 was good, Gnome 3 was where it all went wrong, but now people are switching back to Gnome 3.

See also evilwm (joke) or xfce (supposed to be fast).

When Ubuntu's window manager went bad (11.04 -> 11.10)

(source)

Web developer browser plugins

Chrome:
  • Chrome itself doesn't require admin privileges to install
  • Chrome has dev tools built in - type F12 and select a tab e.g. Network, or right-click on the page and choose 'Inspect element'
  • chrome://net-internals/
  • Web developer plugin
  • Modify headers
Internet Explorer:
  • Internet Explorer should be installed by default
  • It has some dev tools built in - type F12 and then:
    • Menu: Find | Select element by click
    • Menu: Cache | View cookie information
    • Tab: Script | Console
    • Tab: Network (IE9 and later) | Go to detailed view | Double-click an item for headers and cookies
Standalone:
Non-GUI:
  • curl -D -
  • wget -S

Use Data::Dumper with expensive data in perl

$logger->debug(sub { "large data structure is: ".Dumper( expensive_calculation($foo) ) });

This ensures that expensive_calculation() is not performed unless the log level is DEBUG or below.

Open source code review tools comparison

Options I'm familiar with for Subversion:
  • ReviewBoard - good, but fiddly to install. Emails all your comments in a batch when you're ready.
  • Atlassian Stash - okay but generates one email per comment. No, that's been fixed!
  • Kallithea - looks good, not tried.

Why use virtual containers?

Traditional software development:

  • Business need
  • Approval process   }
  • Hardware purchase  } Use IaaS instead - Infrastructure as a service
  • Software development
  • Deployment         } Use PaaS instead - Platform as a service
  • Feedback

The future:

  • Linux containers AKA Docker
    • Security/compliance are happy
    • Operations are happy
    • Business is happy
    • Developers are happy
  • Kubernetes
    • June 2014: Google open source container management project
    • Red Hat is collaborating

Source: Jeremy Brown of tenfourty.com / @tenfourty

What good Perl looks like

This is a selection (not an exhaustive list) of points which make Perl code a joy to maintain:

QUALITY
    * Named parameters to subroutines, passed as a hashref. Some cases of subroutines taking only one or two positional parameters may be acceptable, if it's obvious what they should be.
    * Parameters to subroutines are validated (e.g. Params::Validate, MooseX::Params::Validate, Type::Tiny with Moo
    * use strict; use warnings; # or equivalent (e.g. Moose or NAP::policy) for every file. A test to ensure this, e.g. Perl::Critic's RequireUseStrict and RequireUseWarnings.
    * Does not use the Switch module, the bugs in that module are extremely dangerous and even affect code which doesn't appear to use switch at all.
    * Well factored - write several subroutines or subclasses with meaningful names and avoid large "if/then" blocks.
    * Semantic classes, not utility classes
    * Modules have @EXPORT_OK (instead of @EXPORT), so imported methods have to be declared whenever used, and it's easy to trace where they came from
    * Variable names are full words with underscores separating words, no abbreviations just "to save typing"
    * No magic numbers or magic strings, use constants with comments explaining them. Even better, encapsulate in a method so constants don't have to be defined everywhere.
    * Do not re-use variables, i.e. don't use them for different purposes in different parts of the code. Use two variables instead.
    * Code written in the language of the problem domain (e.g. $album->{artist}) and not the solution domain (e.g. $hash->{lookup_field}).
    * Never perform an eval without checking the error and re-throwing the exception if it's not recognised
    * Code written to be easily understandable by other developers. No cleverness or "magic" without detailed explanatory comments.
    * Logging via at least one abstraction layer so log output can be easily controlled. Ideally use Log::Any which implements the Observer pattern, so you don't have to pass a log object into your classes.

DOCUMENTATION
    * Comments to explain the intention of every distinct section of the code
    * At least a few words of POD for _all_ classes and methods explaining the reason for existence, and what it represents in the real world
    * POD for functional tests (not needed for unit tests)
    * Everything built with a developer in mind who has never seen the system before (there will be many of these in the software's future).

TESTS
    * Test suite can run anywhere (checkout dir, dev env, test env, etc.)
    * Test suite not brittle, tests pass when run in any order
    * Tests clearly separated into:
        * environment - if these fail, bail out
        * unit tests - only test one module, with dependencies mocked out
        * integration tests - tests involving multiple in-team systems
        * monitoring - "tests" of external systems - not really tests but a form of monitoring
    * Appropriate mix of unit and integration tests
    * Tests for POD, syntax, etc. even if with a blacklist

ARCHITECTURE
    * No logic in the templates, proper MVC separation
    * No logic in scripts, only in re-usable modules

FORMATTING
    * No mixed tabs and spaces
    * Indent with 4 spaces
    * Consistent indenting
    * No indenting for longer than about 50 lines. If that happens, break some of the logic out into a different subroutine.

The opinions above are my own.

A nod of the head to Damian Conway's "Perl Best Practices".

Perl modules for SOAP

Options/notes:
  • W3C::Soap - not good
  • SOAP::Lite - not as good as XML::Compile
  • XML::Compile - recommended
    • XML::Compile::SOAP
    • XML::Compile::SOAP11
    • XML::Compile::WSDL11
...but generally SOAP isn't much fun.
Expect the WSDL to be wrong, you may need to store a "fixed" copy locally.

How to install ruby, rake, rubygems

https://www.ruby-lang.org/en/documentation/installation/

For ruby, tried:
  • https://rvm.io/ - wouldn't work on my Suse setup
  • https://github.com/sstephenson/rbenv#readme - worked on Suse. Promising, I like.
    • https://github.com/sstephenson/ruby-build#readme - doesn't work


For rubygems, tried:
  • https://rubygems.org/pages/download - worked

For rake:

  • It comes with ruby


Now learn Ruby:
https://github.com/neo/ruby_koans


Search engines

List:

  • HP Autonomy IDOL
  • Google search appliance
  • Solr / Lucene
  • Atlassian Jira/Wiki search
  • Knowledge bases (untested):
    • http://www.helpgizmo.com/
    • https://www.zendesk.com/knowledge-base
    • http://www.knowledgebase-script.com/
    • http://www.spiceworks.com/free-it-knowledge-base-software/

Which data file format?

List:

  • CSV
  • TSV
  • INI
  • XML
  • YAML
  • JSON
  • RDB
  • custom

iPhone: No new pictures or videos were found on this device

(Windows) Sometimes photos get "stuck" on the iPhone, and the usual method of right-clicking on the iPhone in My Computer and importing doesn't work. It errors: "No new pictures or videos were found on this device".

The fix is to rename the file C:\Users\[username]\AppData\Local\Microsoft\Photo Acquisition\PreviouslyAcquired.db

NOTE: AppData will probably be hidden - you must make hidden files and folders visible in Windows Explorer options.

(source)

Prevent Windows from restarting after an update

Windows Vista/7: Start > search for gpedit.msc. Navigate to Computer Configuration > Administrative Templates > Windows Components > Windows Update and enable "No auto-restart for scheduled Automatic Updates installations".

All non-Home users can apply the same policy change by adding a new key to the registry. Go to Start > Run/Search for regedit. Navigate to HKEY_LOCAL_MACHINE SOFTWARE Policies Microsoft Windows WindowsUpdate AU. Create a new 32-bit DWORD value named NoAutoRebootWithLoggedOnUsers and give it a value of 1.

(source)

Oracle commands for MySQL/PostgreSQL users

Oracle tips for MySQL users:
  • Simple range: SELECT * FROM (SELECT * FROM foo) WHERE ROWNUM <= 100; -- equivalent of MySQL's LIMIT clause, only for start of table
  • Wrong range: SELECT * FROM (SELECT rownum r, f.bar FROM schema.foo f) WHERE r > 100 AND r <= 200; -- Selects a range, but order will be inconsistent, even if you add ORDER BY to inner select
  • Right range:
    SELECT * FROM (
        SELECT q.*, ROWNUM r FROM (
            SELECT * FROM schema.foo ORDER BY id
        ) q
    ) WHERE r >= 100 AND r < 200
    (source)
  • SELECT last_name FROM employees WHERE last_name LIKE '%d_g\_cat%' ESCAPE '\';
    • matches dog_cat, foodig_catbar, etc.
    • _ = any single character
    • % = any characters
    • \_ = literal _ (underscore)
    • ESCAPE '\'; -- set the escape character to \ (backslash)
  • SELECT ...... WHERE REGEXP_LIKE (instance_name, '^Ste(v|ph)en$');
  • ALTER USER foo IDENTIFIED BY "newpassword456!" REPLACE "oldpassword123!"; -- change password (REPLACE is new in 9.2)

Programming principles

* SOLID
* YAGNI
* KISS
* DRY
* TDD
* small methods, one level of abstraction per method (also methods over branches). No snappy abbreviation for this one :(

Programming quotes

Defensive code is the difference between fragility and robustness --gloryhack

Outlook 2007: Don't remove line breaks

Microsoft :'-(

Outlook 2007:
Tools | Options | Preferences | E-mail Options
Untick "Remove extra line breaks in plain text messages"

(source)