A piggy bank of commands, fixes, succinct reviews, some mini articles and technical opinions from a (mostly) Perl developer.

See hidden characters in a string

perl -e '$HOSTNAME = `hostname -s`; print $HOSTNAME;' | od -c

Easily add colours to bash scripts

RED="$(tput setaf 1)"
GREEN="$(tput setaf 2)"
YELLOW="$(tput setaf 3)"
WHITE="$(tput setaf 7)"
RESET=$WHITE

echo "${RED}An error occurred${RESET}"
echo "${YELLOW}Warning: Be careful${RESET}"
echo "${GREEN}Everything is groovy${RESET}"

(source)

Old, naive way:

GREEN="\[\033[32m\]"
YELLOW="\[\033[33m\]"
RESET="\[\033[0m\]"

Outlook 2007 reminder windows always on top

Hit Alt-F11, navigate to Project1 | Microsoft Office Outlook Objects | ThisOutlookSession

Simple way 

Private Sub Application_Reminder(ByVal Item As Object)
    If TypeOf Item Is AppointmentItem Then
    MsgBox "Message text", vbSystemModal, "Message title"
    End If
End Sub


(source)

Sophisticated way

Private Declare PtrSafe Function FindWindowA Lib "user32" _
(ByVal lpClassName As String, ByVal lpWindowName As String) As Long

Private Declare PtrSafe Function SetWindowPos Lib "user32" ( _
ByVal hwnd As Long, ByVal hWndInsertAfter As Long, _
ByVal X As Long, ByVal Y As Long, ByVal cx As Long, _
ByVal cy As Long, ByVal wFlags As Long) As Long

Private Const SWP_NOSIZE = &H1
Private Const SWP_NOMOVE = &H2
Private Const FLAGS As Long = SWP_NOMOVE Or SWP_NOSIZE
Private Const HWND_TOPMOST = -1

Private Sub Application_Reminder(ByVal Item As Object)
Dim ReminderWindowHWnd As Variant
On Error Resume Next
ReminderWindowHWnd = FindWindowA(vbNullString, "1 Reminder")
SetWindowPos ReminderWindowHWnd, HWND_TOPMOST, 0, 0, 0, 0, FLAGS

End Sub

(source)

Email me that information, don't tell me out loud!

Why I want you to email me that information instead of telling me out loud. There are so many reasons:

Speaking is inefficient:

  • I could mis-hear and copy it down wrong
  • I have to spend time listening to you and writing it down somewhere
  • If I'm not in front of a computer then it's written on a piece of paper, I have to store it somewhere for future reference, or spend more time typing it up

Email is efficient:

  • You've got a record of the information for the rest of your life, with no further effort (you are using a web-based email system, right?)
  • You can search for and find the information instantly
  • You can forward it to someone else for their reference
  • You can copy and paste with no chance of error
  • Email is so fast that someone can email you while you're speaking on the phone, and you can receive it and verify that it's correct before they hang up


STDERR and STDOUT appear in the wrong order when piping to a file

a Perl script - test.pl:

print "OUT 1\n";
print STDERR "ERR 2\n";
print "OUT 3\n";
print STDERR "ERR 4\n";
print "OUT 5\n";

Run the script, piping all output into a file:

perl test.pl &> file.log

cat file.log

ERR 2
ERR 4
OUT 1
OUT 3
OUT 5

Output is in the wrong order :(

Run it again using unbuffer:

unbuffer perl test.pl &> file.log

cat file.log

OUT 1
ERR 2
OUT 3
ERR 4
OUT 5

Output is in the correct order :)

(source)

Another way:

script -c 'perl ~/temp/stream.pl' file.log

cat file.log

(source)

Cannot scp, but ssh works fine

Maybe your .bashrc or similar prints something to the console? This breaks scp.

e.g. if .bashrc or something it calls contains:
echo "Welcome to the server"

Then scp will just print "Welcome to the server" and not even attempt to copy the file.

Can't install recent Perl: too few arguments to function ‘dbmclose’

When I tried to install perl 5.16.2 or perl 5.20.0:

wget http://www.cpan.org/src/5.0/perl-5.20.0.tar.gz
tar -xpvzf perl-5.20.0.tar.gz
cd perl-5.20.0
sh Configure -Dprefix='/opt' -de
make
...
make[1]: Entering directory `/opt/dweb/home/shepwil/src/perl-5.20.0/ext/ODBM_File'
cp ODBM_File.pm ../../lib/ODBM_File.pm
Running Mkbootstrap for ODBM_File ()
chmod 644 ODBM_File.bs
../../miniperl "-I../../lib" "-I../../lib" ../../lib/ExtUtils/xsubpp -noprototypes -typemap ../../lib/ExtUtils/typemap -typemap typemap  ODBM_File.xs > ODBM_File.xsc && mv ODBM_File.xsc ODBM_File.c
cc -c   -fwrapv -fno-strict-aliasing -pipe -fstack-protector -I/usr/local/include -D_LARGEFILE_SOURCE -D_FILE_OFFSET_BITS=64 -O2   -DVERSION=\"1.12\" -DXS_VERSION=\"1.12\" -fPIC "-I../.."   ODBM_File.c
ODBM_File.xs: In function ‘XS_ODBM_File_DESTROY’:
ODBM_File.xs:128: error: too few arguments to function ‘dbmclose’
make[1]: *** [ODBM_File.o] Error 1
make[1]: Leaving directory `/opt/dweb/home/shepwil/src/perl-5.20.0/ext/ODBM_File'
Unsuccessful make(ext/ODBM_File): code=512 at make_ext.pl line 561.
make: *** [lib/auto/ODBM_File/ODBM_File.so] Error 25

The solution:

sudo diff /usr/include/dbm.h.patched /usr/include/dbm.h
62c62,63
> extern int    dbmclose __P((DBM *));
---
< /* extern int dbmclose __P((DBM *));*/
< extern int    dbmclose __P((void));

sudo cp usr/include/dbm.h.patched /usr/include/dbm.h

(source)

How to communicate within an IT department or tech team

Communication is difficult, it's easy to get wrong. Here are some ideas about how to get it right.
  • Mailing lists: Make them public (within the company). Archive them. Make them discoverable. Let users administer them. See GNU mailman.
  • Wiki: Only have one. Keep the software up-to-date. Have an 'information architect' role, who manages the structure, so that the information is in predictable places. Make sure the search works (test it regularly). Unlock all the permissions. Let people delete pages. React to user feedback.
    • When you upgrade, ideally migrate/import the data to the new version. If that doesn't happen, then export or otherwise safely archive the old content. Publish it internally, make sure it's available, even if static.
    • Use the Gliffy plugin or equivalent (e.g. on Atlassian Confluence an open source wiki) or equivalent to allow users to publish diagrams of unlimited complexity, that can be edited and kept up-to-date by anyone.
  • Issue tracking system: Keep the software up-to-date. Don't install a million plugins (looking at you Jira) that make it difficult to upgrade. Make sure the workflows match reality, give people standard ones and document how they work. Be very careful when making changes to workflows, try to keep them as general and re-usable as possible.
  • Write high quality tickets.
  • Service Desk: Use the same system as the bug tracker, which will allow all staff to track their tickets' progress in a way they understand. If this is not possible, make the service desk ticket system no less usable/visible than the staff issue tracker system.
    • Ensure the service desk issue system is no less usable than email, i.e. original message chain quoted in all replies, and CCs honoured with reply-all.
    • Have a special email address for people to CC which will copy the email to a ticket. Publicise it regularly.
  • Choose software that makes a healthy development environment.
  • Things to publish internally, and regularly re-post:
    • instructions for transferring phonecalls internally
    • instructions for setting up a conference calls (for all phone models in use)
    • instructions for setting up video calls from conference rooms
    • instructions for setting up a laptop/desktop with the big screen in meeting rooms/presentations
  • List all regular non-stream tasks, define roles to triage, co-ordinate and communicate them to the rest of the team, with a view to improving and streamlining the work of the team. Examples:
    • Failing tests
    • Environmental issues
    • Live bugs
    • Supporting the release
    • Inter-team liaison
    • Monitoring reports
    • Business investigations
  • Ideally don't have any remote members of the team. But at the very least, don't inconvenience the co-located members for the sake of remote members. This means:
    • Don't hold scrum via phone or skype, hold it in person. Figure out some way for the remote people to attend that doesn't take anything away from the people in the room. If there's no way, they can attend their own scrum, or not at all.
      • Update: Video chat (e.g. Zoom) works well when there is a large TV screen and dedicated room microphone device. Whoever is in the office goes over to that TV & mic area and starts the video call so anyone working remotely can dial in and participate.
    • Don't force people to communicate via headsets when they're standing next to each other. That would discard the many benefits of face-to-face communication.
    • Make sure everyone actually stands up around a physical board (Update: Jira scrum/kanban board works fine over video chat, if updated outside the meeting). If headset range is too far to work, don't use them.
    • Don't have the scrum at a weird time of day to accommodate other time zones.
    • If it's difficult to hear the remote members then have them email points to the team.
    • Try to arrange for the remote members to work one-on-one with each co-located member, to build rapport
  • Put ALL work in the issue-tracking system to make it visible, even non-development work. If it's not tracked, you didn't really do it. Each task must link back up to team "epics" or long-term goals, and ultimately to a company goal. Must be extremely easy to create tasks with one line of text and no other info, i.e. prevent unnecessary overhead for simple tasks.
  • Specific to large companies:
    • Instant chat client for ALL members of staff, with zero configuration, it's just there waiting to be used. Hooked into Outlook and the O/S so that it knows when you're in a meeting or away from your desk and updates your status accordingly (e.g. MS Office Communicator)
    • Have a meeting room booking system that is painless (is this even possible?) and transparent with regard to: higher-ranked employees permission to "override" bookings, policy for block-booking rooms every week, etc. Above all it should be obvious what is happening and why.
  • IRC/Slack or equivalent for all. Make it clear that all communications will be logged.
  • (crazy idea) Employ a person/team whose sole remit is to make it easier to communicate within the company. This is definitely not the usual "Internal Communications" team that sits with HR and serves the Executive. It's a team that is there to support IT workers.
  • If the office is very large, publish a map of where each team sits. Make sure it's always up-to-date, either by giving a team the responsibility to do it, or making it editable by anyone on the wiki.
  • Link from the team map to official wiki pages, etc. org chart, make everyone's names and roles discoverable and visually represented.
  • Put signs up around each area so you can see which team sits where.
  • Use scrumblr.ca or funretro.io or similar (virtual post-it note boards) for retrospectives, to allow ideas to be captured throughout a sprint, not just on the day. Also a wiki page does just fine. But it's also important to keep real, physical post-it notes and pens in the meeting for people to jot down ideas they just think of at that time.
  • Create the following types of wiki pages, for each team or development group:
    • Tech Debt / Yak Shaving / Infrastructure to-do list / Ideas / Suggestions (use your judgement if this should be one page or several). This is a list of things that people want to do, changes they want to make that are not on the roadmap, or don't obviously provide direct value to the business. The team may need help explaining where the value lies.
    • Proposed technical discussion topics. Topics worthy of group discussion. Schedule a regular meeting (every 2-4 weeks) to discuss whatever is on this list and make a decision as a team about how to proceed.
    • Developer FAQ. Either per-team or per-department.

Source: My own experience working in IT.

What a healthy development environment looks like

In my opinion, the best set up would include:

Software
  • Jira for issue tracking 
  • A wiki for information sharing 
  • Git for version control
  • Continuous integration and continuous deployment (e.g. with Jenkins)
  • Use an existing package provider like Redhat, don't compile publicly available software yourself
Environment
  • Full commit history available with details of developer name and change ticket ID
  • Full ticket history available to read, with no historical tickets in the "old" system and unreadable
  • /etc/init.d scripts ('service') for starting, stopping and restarting software
  • VMs for development 
  • Identical production, development and test environments
  • Continuous Integration: Every check-in to any development branch is run through the test suite, and any failures are emailed to the author
  • Immutable infrastructure: Servers are not changed. All changes begin with pushing to a repo, and a new server is then built and deployed to a subset of users, tested, then to all users.
  • One-touch deployment, identical in both production and development environments 
    • Continuous deployment: Every commit to trunk that passes the test suite is deployed
  • Refactoring code as required, restricted only by the need to pass the test suite
  • All major software components described on wiki with details of source code, restarting, troubleshooting, owner, etc.
Tests
  • Easy to run the entire test suite for any branch
  • Test suite should complete quickly
  • A test suite that is comprehensive and well understood by all
  • Test coverage stats are visible to everyone
  • Tests for hallmarks of quality like 'use strict' and embedded documentation (even if a blacklist of legacy code is required)
Code
  • Clear model-view-controller separation
  • All code contained within re-usable modules (scripts are thin wrappers around classes)
  • No "utility classes" (inherit/use roles if common methods are needed)
  • No "god classes" (split up methods into meaningul sublclasses or roles)
  • All logic contained within meaningful "noun" classes designed for problem domain
  • Meaningful variable names in domain language and no unnecessary abbreviations
  • Plentiful comments, written in the language of the problem domain
  • All classes/modules to have embedded documentation
People
  • For every piece of work, three people should discuss it together for a few minutes after a technical design has been proposed, and before any development work is started - a representative of the business, a QA tester and a developer.
    • They should ask questions like "Are the requirements accurate and high quality?", "Does the proposed solution fulfill the requirements?", "Is the proposed solution high quality?", "Can the proposed solution be easily tested?", etc.
  • The product manager as well as the people coming up with requirements must be readily available to answer questions.
  • To facilitate communication. teams should *not* be split across timezones
    • (separate) Scrums should take place at the beginning of the local day

example gdb command

Use gdb to get a backtrace by attaching to a running process, then detaching immediately like so: sudo gdb /path/to/apache/bin/httpd 12345 -ex bt -ex detach -ex quit Where 12345 is a process ID that is misbehaving. This command should be used with a bit of care as it will pause that process while the backtrace is obtained.

How to forward your SSH key

Windows:

1. Run pageant, put it in startup, and pass it your key

2. Run putty or MobaXterm, configure them both to do "ssh -A" (i.e. forward your key)

3. On the server, check your SSH key is loaded:

ssh-add -l

4. ssh -A to other servers

Linux:

ssh -A to each server