JJ5-bin
jj5-bin is where John keeps his Linux scripts. For other projects see projects.
NOTE: I'm sorry but the information on this page is well out of date. It's on my list of things to review, but not with high priority.
Install like this:
curl https://svn.jj5.net/svn/public/jjrepo/jj5-bin/branches/0.3/install.sh | bash
Status
Released, but there's stuff TODO.
Administration
Contributors
People who have contributed to this project. Newest on top.
Copyright
Copyright 2012, Contributors.
License
Licensed under the New BSD license.
Resources
Source code
The repository can be browsed online:
The code is available via Subversion from here:
Links
Specifications
Functional specification
Utilities
archive
Moves the files specified as arguments to an archive directory. Your archive directory is in ~/archive and it should only be accessible by you. Archive directories are named based on the date in yyyy-MM-dd-HHmmss format.
archive-copy
Copies the files specified as arguments to an archive directory. Your archive directory is in ~/archive and it should only be accessible by you. Archive directories are named based on the date in yyyy-MM-dd-HHmmss format.
backup-mysql-db
Backs up a MySQL database on the local host. Accepts two arguments. The first argument is the backup type, typically one of 'hourly', 'daily', 'weekly', 'monthly' or 'adhoc'. The second argument is the name of the database to backup.
Database backups are stored in the /var/backups/mysql/$database_name/$backup_type directory which is only accessible by root.
In order to run the database backup script needs access to the file /home/jj5/.passwd/mysql_root which must contain a single line containing the password for the MySQL root user.
backup-svn-repo
Backs up a Subversion repository. Accepts two arguments. The first argument is the name of the repository being backed up, e.g. 'bkrepo', 'pcrepo', 'jjrepo', etc. The second argument is the file path to the repository, e.g. /var/svn/export/svn/public/jjrepo.
Subversion backups are stored in the /var/backups/svn/$repo directory which is only accessible by root.
bkts
The 'bkts' command is short for 'Blackbrick Timestamp'. A Blackbrick Timestamp is in the form yyyy-MM-dd-hhmmss using 24 hour time and in the Australia/Sydney timezone.
To get a Blackbrick Timestamp run the following command:
$ bkts
To create a directory in the current directory named after the current date/time run the following command:
$ bkts md
To create a directory in the current directory named after the current date/time and to automatically pushd into that new directory run the following command:
$ bkts pd
Note that after running 'bkts' the $TS environment variable is updated with the latest/last timestamp.
See bkts technicalities for technical details concerning implementation.
bkxfer
The 'bkxfer' command is for Blackbrick staff who have access to the Blackbrick bkwork repository. This command will pushd into your user transfer directory. Typical use would be:
$ bkxfer $ bkts md
That would put you into your transfer directory and then create a timestamped directory for you to put your files in.
See bkxfer technicalities for technical details concerning implementation.
ckdir
The 'ckdir' command is a contraction of 'cd' and 'mkdir'. The command issues a 'mkdir' command to create a new directory, and then a 'pushd' command to change directory into the newly created directory.
See ckdir technicalities for technical details concerning implementation.
cron-configure-backups
Installs the hourly, daily, weekly and monthly cron backup scripts for a host that has said scripts available in the lib directory.
fw-restart
Reloads the iptables firewall rules from the standard location /etc/iptables.up.rules.
fw-stop
Disables the iptables firewall allowing (and logging) all traffic. This is in case you need to do some debugging, but use with caution.
keep-only
Keeps only the specified number of files (typically backup files) in a directory. Accepts two arguments. The first argument is the path to the directory being processed. The second argument is the number of files that should be kept.
If the number of files in the specified directory exceeds the number of files to be kept, then the oldest files are deleted so only the specified number of files remain.
ls-all
Lists all files in a directory, or if no argument is given, all files on the system. This script is used by the backup scripts to get a full list of files and their user/group/permissions.
pd
The 'pd' command is an alias for the Bash built-in 'popd'. It pops the top directory off $DIRSTACK returning you to the directory that is the new top of the stack.
See pd technicalities for technical details concerning implementation.
reconnect
Reconnects your screen session.
take-ownership-jj5
Grants ownership of an entire directory (recursively) to jj5:jj5 and makes sure the files are owner accessible.
colorize-web-logs.sh
The colorize-web-logs.sh (download) script works in tandem with customised Apache2 web logs and lib.sh to colourise output. To work with this script you need to copy the config file x-log.conf (download) into /etc/apache2/conf-available/:
# cp x-log.conf /etc/apache2/conf-available/
Then enable the custom log config:
# a2enconf x-log
Then configure your virtual hosts to use the custom config (called "sm_log") by adding these lines or similar to your config files in /etc/apache2/sites-available/:
ErrorLog ${APACHE_LOG_DIR}/error.log
CustomLog ${APACHE_LOG_DIR}/access.log sm_log
Then restart Apache:
# apache2ctl graceful
Next you need to download your web logs in an appropriate fashion so that you can colourise them, here is a bare bones script to do that:
main() {
# 2018-03-05 jj5 - prepare our working directory...
#
quiet rm -rf /tmp/web-spy;
quiet mkdir /tmp/web-spy;
quiet pushd /tmp/web-spy;
# 2018-03-05 jj5 - disable the console screen saver (optional)...
#
setterm -blank 0 -powerdown 0;
# 2018-03-05 jj5 - monitor web logs for hosts 'trust', 'orac' and 'tact'...
#
web_spy 201 jj-srv-1 J trust;
web_spy 202 jj-wrk-1 T orac;
web_spy 203 jj-wrk-2 T tact;
# 2018-03-05 jj5 - we're racing the creation of the log files so give them
# a chance...
#
sleep 3;
# 2018-03-05 jj5 - now that our weblogs are being downloaded into
# /tmp/web-spy (the current directory) we can tail them locally and pipe
# into our colouriserer...
#
tail -f *.log | /home/jj5/bin/colorize-web-logs.sh;
return 0;
}
# 2017-07-06 jj5 - If you need a copy of this library:
# 2017-07-06 jj5 - SEE: jj5-bin standard library:
# https://svn.jj5.net/jjrepo/jj5-bin/branches/0.2/bin/inc/lib.sh
source /home/jj5/bin/inc/lib.sh;
Note: the above script relies on the web_spy() function from the lib.sh library in addition to its runtime helpers to support e.g. the main() function.
Now you can run the above script to colourise your web logs.
Have fun!
Session management
session-info
Prints info about your session that is being recorded.
session-login
The login script run when creating a new recorded session.
session-replay
Replays a recorded session. Accepts one argument, being the path to the session to replay without its file extension (either .timing or .script).
session-start
Starts a new recorded session.
session-stop
Stops recording the current session.
Configuration management
makeslave
The makeslave.sh script calls the makeslave.js script which builds a config file for the slave DNS server in /home/jj5/bin/bind/slave.conf.
Project management
bkrepo-release
Releases a Blackbrick Development project.
pcrepo-release
Releases a ProgClub project.
jjrepo-release
Releases a project in John's public repository.
jprepo-release
Releases a project in John's private repository.
svn-release
Releases a project in a Subversion repository. Accepts three arguments. The first argument is the name of the project being released. The second argument is the release notes. The third argument is the repository root URL.
This script is called from the bkrepo-release, pcrepo-release, jjrepo-release and jprepo-release scripts.
sixsigma
devotion
empathy
orac
Technical specification
The facilities available in jj5-bin are generally Bash shell scripts, functions or aliases.
bkts technicalities
The bkts command is implemented as a function in .bashrc. It temporarily sets the timezone to 'Australia/Sydney' and uses the 'date' command with a format string to build a timestamp.
bkxfer technicalities
The bkxfer command is implemented as a function in .bashrc. It just uses the Bash built-in command 'pushd' to change directory into a user's bkwork xfer directory.
ckdir technicalities
The ckdir command is implemented as a function in .bashrc. It issues a 'mkdir' command and if that is successful a 'pushd' into the newly created directory, returning a zero error level on success or non-zero error level on failure.
pd technicalities
The pd command is implemented as an alias in .bashrc. It just maps the new 'pd' command to the existing Bash built-in command 'popd'.
Notes
Notes for implementers
If you are interested in incorporating this software into your project, here's what you need to know:
Just copy-and-paste your way to victory! Feel free to ask questions.
Notes for developers
If you're interested in contributing to this software, here's what you need to know:
Send your patch to jj5@jj5.net along with a statement that you are willing to be listed in the contributors section of the documentation and willing to license your contribution under the license.
For security reasons write access to jjrepo is only available for John. It's basically my private repository. If you'd like to hack on projects where you do have checkin permissions on the Subversion repository, along with infrastructure for running your code, then why not head over to ProgClub and register?
Release notes
Release notes are (supposed to be) added after notable releases are made. New release notes are added to the bottom of the release notes section and have a level 3 title based on the release date.
Tasks
TODO
Things to do, in rough order of priority:
- implement the stub scripts (e.g. session-info).
Done
Stuff that's done. Latest stuff goes on the bottom of the list.
2012-03-01
- JE: created project page and initial documentation.