Technical information I have found useful

Topics

Internet

The best (subjective), free and zero ads (objective) Internet Speed Test (see help info) comes from Google, or the Google partner direct link.

Private DNS Resolver, 1.1.1.1

DNS encrypt proxy with adblocker for Linux, www.cyberciti.biz

If you want to know your browser info and IP address, ISP and location info try one of these for varying levels of detail
To scan a network and find all equipment on it. Ensure IP address pattern matches your network, these tend to be 192.168.0.1 for EU, 192.168.1.0 for US etc:
To check the network traffic of your device:

Chrome Browser

How to setup Chrome Tab Groups to open multiple pages simultaneously

I used to use the Settings / Save Open Pages option, but Tab Groups lets you do multiple different sets. A colleague, Neil Fisk, wrote the following excellent instructions to help me set these up.

Chrome has a feature called Tab Groups. This is where you can group tabs together according to requirements. I would suggest this is what you're looking to do. Below is a guide to setting them up so that you can recall them whenever you start Chrome for the first time.
Open all of the sites that you want to have open when you start Chrome. Right-click the first tab and select "Add to tab group". Give the group a name (i.e. Startup).
Now, right click on each of the remaining tabs and add them to the group you've just created.
Once done, all of your startup tabs are grouped together. Make sure you don't have any other tabs open which you don't want included in the group. Then, click the vertical three dots “more” menu at the top-right of the Chrome browser and hover over “Bookmarks”. From there, you’ll see a “Bookmark all tabs” option.
Once the dialog box appears and asks you where you’d like to save all of these tabs collectively, just choose the Bookmark bar for now. Here’s the important part – whatever you do, don’t click the “New folder” button on the bottom-left. All of these grouped tabs will automatically be placed into a Bookmark folder by the name of the Tab Group they came from, so all you’ll need to do is to fill out the parent folder name found in the “Name” text box of the pop-up dialog.
Now, just finish up by clicking the Save button on the pop-up dialog, and then you’ll see that it has been added to the Bookmarks bar, or wherever you’ve saved it to. You are now free to right-click a Tab Group and select “Close group” to free up your workspace without fear of losing anything.
To recall your tabs as the same group, you can right-click the Bookmark folder on your bar or in the Bookmark Manager (Ctrl+Shift+O) and choose the “Open all in new tab group”. Immediately upon doing so, you’ll see your Tab Group reappear with the same name, color, and tabs that you last left off with.

YouTube

To set a time point to on youtube video, append &t{time}{units} for example to set a video 875 seconds into the beginning append &t=875s to the URL, e.g. https://www.youtube.com/watch?v=Zx2xYWrFKR8&t=875s
To download and keep youtube videos I recommend:
Audio Only Extract from YouTube

If there is a song you like, but only the video is out, then use the youtube-dl to download the audio elements, e.g. 


      youtube-dl -k https://www.youtube.com/watch?v=AWhHUaEfecc
then, look for the file(s) downloaded and convert for your MP3 player 

      ffmpeg -i thevideo.mkv -ab 320k thevideo_audiotracks.mp3
Or only using youtube-dl to get an mp3. Also shown is how to use ffmpeg to skip the beginning (in the example 4.1 seconds),
in case there is some junk there you do not want 

      youtube-dl -x --audio-format mp3 --audio-quality 320k https://www.youtube.com/watch?v=AWhHUaEfecc
      ffmpeg -i youtube.mp3 -ab 320k  -ss 00:00:04.100 skipped_song.mp3
or 

      yt-dlp -x --audio-format mp3 --audio-quality 320k https://www.youtube.com/watch?v=HR36AQVZ10k
Best Video with Best Audio Extract from YouTube

If there is a video where there is a high quality video but has no sound in that format, you need to combine the video with one of the other audio streams. I wrote a small script to enable this, see youtubedlb.sh (download)
Examples: 


      ./youtubedlb.sh https://www.youtube.com/watch?v=EDwb9jOVRtU
      ./youtubedlb.sh -m https://www.youtube.com/watch?v=EDwb9jOVRtU
using yt-dlp, to force a specific format, e.g. mp4 with the best quality video and audio 

      yt-dlp -f "bestvideo[ext=mp4]+bestaudio[ext=m4a]/best[ext=mp4]/best" https://www.youtube.com/watch?v=rRaVGKk4k6k
MP3 Meta Data

If you need to change the meta data in the mp3 files my fav tool is id3v2.
Examples: 


    id3v2 --TPE2 "Various Artists" *.mp3
To change all the artist in all tracks to "Various Artist"

Linux Utils

Auto Clicking the Mouse

A very powerful (and dangerous) tool, xdotool allows you to simulate mouse and keyboard presses.
You might need to install with sudo apt-get install xdotool

The cursor MUST BE EXACTLY where you need it when the clicking starts and all the time it continues.

Examples:
xdotool click --repeat 100 --delay 18 1

When you want to stop it, press and keep pressed the mouse button so that no more clicks are registered, and Alt-Tab to the console window to stop it with Ctrl-C

To click every 30 seconds for 1 hour, with 5 seconds delay before the cycle starts so you can get the mouse in place
sleep 5 ; xdotool click --repeat 120 --delay 30000 1

Tracking files in dir

Tracking files created in a dir, shell script is amazing and powerful, with a simple one line command you can track files created in a directory and see what the updated contents is of the latest file to be updated
while : ; do AF=`ls -rt | tail -1`; tail -20 $AF; sleep 2 ; done
Finding biggest files

In case you are running low on space, finding the biggest files is very easy under Linux, below will bring back top 10 biggest files on a drive, adjust the path and count at the end to see different paths and numbers thereof
find /mnt/mydrive/start.point -type f -print0 | xargs -0 du -h | sort -rh | head -n 10

Video Capture

Some notes on capturing videos playing in the screen. Below you see: checking on devices, capture (while playing in fullscreen), trimming (to cut out the capturing of me running the commands), extracting the audio.
pactl list short sources ffmpeg -video_size 1920x1200 -framerate 30 -f x11grab -i :0.0 -f pulse -i alsa_output.pci-0000_00_1b.0.analog-stereo.monitor -c:v libx264 -preset ultrafast -c:a aac output.mp4 ffmpeg -i output.mp4 -ss 00:00:31 -c copy trimmed_output.mp4 mv trimmed_output.mp4 chatgpt-song.mp4 ffmpeg -i chatgpt-song.mp4 -vn -acodec mp3 chatgpt-song.mp3

If you're wondering about the naming, I just had to have a local copy of the excellent post of the ChatGPT song created by the Head of AI at Moonpig that was posted on LinkedIn , but they do not allow downloading.

Video Splitting

Why would you need split videos into smaller chunks? Recently I got the full set of Star Wars movies (again, now I have VHS, DVD and 4KBlueRay disks), and I have a 4K TV which can read off a memory stick, so the logical solution is to put all the movies onto a 128GB memory stick plugged into the TV for instant, emergency access ... but a 4K movies is about 8GB and FAT32 formatted drives (which my TV supports and most memory sticks tend to be preformatted to) can only take 4GB as an upper file size limit, so you need to split the big 4K movie file into chunks. Using ffmpeg makes it simple, with no affect on the original quality, and can handle multiple file types such as mp4 and mkv files, as shown below.
ffmpeg -i ORIGINALFILE.mp4 -acodec copy -vcodec copy -ss 0 -t 00:15:00 OUTFILE-1.mp4 ffmpeg -i ORIGINALFILE.mp4 -acodec copy -vcodec copy -ss 00:15:00 -t 00:15:00 OUTFILE-2.mp4 ffmpeg -i ORIGINALFILE.mp4 -acodec copy -vcodec copy -ss 00:30:00 -t 00:15:00 OUTFILE-3.mp4

To make life easy, use this python script (local copy here) so you can just use a chunks parameter, so you do not have to worry about time points
python ffmpeg-split.py -f ORIGINALFILE.mkv -c 2
which will produce two chunks (from -c 2) named ORIGINALFILE.1-of-2.mkv and ORIGINALFILE.2-of-2.mkv

For further info see splitting an mp4 file

Additionally, you might want to remove hardcoded subs from mkv files as follows
ffmpeg -i originalvideo.mkv -vcodec copy -acodec copy -sn video-no-subs.mkv

Video Picture Extract

I found some photos of a wedding we attended many years ago, the camera was an early digital one, only 3MP, but the video was quite good (but no sound). The photos taken were really not that good or sharp, but the video looked better, so it made sense to extract some photos directly from the video, though that would be even lower resolution, at least they would be sharper, ffmpeg to the rescue again.

Try varying the rate (-r) to extract more or less frames, the bigger the number, the more frames are extracted.
ffmpeg -i ../videofile.AVI -r 1.5 -q:v 1 -qmin 1 wedding-%3d.jpeg

Another example, starting at 36 mins and 6 seconds into the video, running to the end, the order of the options is important for performance reasons
ffmpeg -ss 00:36:06 -i /mnt/amachine/adir/series/TheSeries/TheVideo.mp4 -r 2.5 -q:v 1 -qmin 1 image-%3d.jpeg

Reminders

My memory is getting worse, and working hard, focusing, I even forget to get up off the seat, meaning my legs feel dead by the end of each day.

doexercise.sh is a useful little script I wrote to remind me to at least do some exercise, it is called via at or cron on a cycle. It will give you a prompt in a little window, and also create a popup which has a random selected exercise from https://musclewiki.org.

To schedule it on Linux, I use a crontab command of 10 10,11,12,13,14,15,16 * * 1,2,3,4,5 ./tools/doexercise.sh (see download doexercise.sh), obviously change the path to where you install and change the schedule, mine is weekdays, Mon-Fri, at 10mins past every hour from 10am to 4pm.

If you need an audio reminder to do something soon, try scheduling a little sound to be played when something is done or needs to be done thus
aplay ~/Downloads/mixkit-happy-bells-notification-937.wav

You can put a sleep or something before this, or sequence the command set with mycommand ; ffplay -nodisp -t 1 -autoexit notes.mp3 2>/dev/null to make it happen once a command has finished.

Security

Just in case you need to delete a file and ensure it cannot be read or recovered, the best way is to shred it, not just delete it, as they can be recovered, but to write to it many, many times, which makes it nearly impossible to recover (unless you have near infinite time or resources).

To shred one file just use shred myfile, but to shred an entire directory tree of files do the following
find <base_directory> -type f -exec shred -u {} \;

or for a specific type of file, e.g. all .csv files
find <base_directory> -name "*.csv" -exec shred -u {} \;

System

To check your local release version
lsb_release -a

To check the upstream release version (the base yours came from)
cat /etc/upstream-release/lsb-release

Linux updates are amazing and are applied when your system is up and running, so no waiting at a blue screen, occasionally older stuff is left hanging around, if you want to tidy this run the following
sudo apt-get install --fix-broken

This will give a list of things you do not need anymore, then run the following command, which will also clean up any unnecessary used space in transient files, expired backups, system images etc
sudo apt autoremove

There is a really good guide on /boot clean up at itsfoss.

If you want to copy or move a bunch of files around, but want to control the rate to limit the impact on your network, you can use rsync with bandwidth control. The following will recursively (the -r) move (the --remove-source-files) files and directories (source dirs will be left empty but present), limiting the rate to 20kb/s (the --bwlimit)
rsync -rav --progress --bwlimit=20000 --remove-source-files /source_base/source_path/* /dest_base/dest_path

To do a faster move, where the file checksum is checked and only updated if different use the options -rcauv

If you need a second monitor but do not have one handy, you can turn a tablet or mobile into one using Deskreen
To setup VirtualBox on a UEFI SecureBoot Linux machine see this page for Ubuntu like distros
To save space, find big files, convert files etc see 7 Linux command-line tips for saving media file space

I wrote a simple script to go through a destination, find and shrink larger mp4 files, see makevideosmaller.sh at Sample Code

Video settings can sometimes be a bit complex on a new system, especially getting the best monitor resolution, see How to change screen resolution for a good set of instructions.
Summary: sudo lshw -short -class display xrandr --listactivemonitors xrandr cvt 1600 900 xrandr --newmode "1600x900_60.00" 118.25 1600 1696 1856 2112 900 903 908 934 -hsync +vsync xrandr --addmode DVI-0 "1600x900_60.00" xrandr --output DVI-0 --mode 1600x900_60.00 cvt 1920 1080 xrandr --newmode "1920x1080_60.00" 173.00 1920 2048 2248 2576 1080 1083 1088 1120 -hsync +vsync xrandr --addmode DVI-0 "1920x1080_60.00" xrandr --output DVI-0 --mode 1920x1080_60.00
Set the resolution at least ONCE using the Display app, next time you login, it will remember.
There are a number of really good tools which you can use to recover disk, one of the best and most flexible can handle most file systems, even if they are very messed up. Below is an example of fsck running on an ext4 disk at /dev/sdd, using a backup super block (819200) and telling it the block size is 4096. fsck.ext4 -f -y -b 819200 -B 4096 /dev/sdd

Productivity and Personalization

I got bored of starting the same apps everyday, so I wrote a small script which takes a JSON config file which allows you to setup a variety of repetetive tasks, take a look at my lsetup.sh script for more details.
There are some great tools to help save space and find duplicates, one I tend to use a lot is Detwinner which is available as a Flatpak at code.neatdecisions.Detwinner. Typical use, to enable you to check local and remote drives, even mounted drives (need sudo to delete on these), do the following (once installed):
sudo -i /usr/bin/flatpak run --branch=stable --arch=x86_64 --command=detwinner com.neatdecisions.Detwinner

Windows on Linux

Wine (Wine Is Not an Emulator), is not an emulator which allows you to run many Windows application directly on Linux, many of my favourite utility applications can run happily under Wine, so if you really need that app, and there isn't a version on Linux (though more and more modern apps are multiplatform), then try to run under Wine. Some useful Wine pages:

Windows Utils

Windows95 Emulator running in JavaScript Electron app (on GitHub)

Reset/Reinstall Windows10

Some good guides on reinstalling or resetting Windows10

Windows Subsystem for Linux

Amazing that Microsoft are now supporting a variety of Linux flavours which can be run from the command line. Great news if you are working in an AWS or other Linux Cloud environment to have this level of capability locally. Of course my dev boxes are all Linux anyhow, but if you are a Windows user this is a real plus. I installed the Ubuntu distribution as it was the closest to my Linux Mint desktops.

Product Key

  1. Press Windows Key + X.
  2. Click Command Prompt (Admin)
  3. At the command prompt, type: wmic path SoftwareLicensingService get OA3xOriginalProductKey
  4. This will reveal the product key.

Dual Boot

How to convert a BitLocked Windows PC into a Dual boot Windows and Linux box

Speeding up Windows 10

I gave up on Windows when we started using AWS at work and found that a Linux desktop worked much better for me, in every way. My kids however, are most interested in their games, which run best or only on Windows.

Finding Info on Installed Programs

Do the following in PowerShell

$loc = Get-ChildItem HKLM:\Software\Microsoft\Windows\CurrentVersion\Uninstall $names = $loc |foreach-object {Get-ItemProperty $_.PsPath} foreach ($name in $names) { Write-Host $name.Displayname }

The command echo $loc can be used to get full info

Productivity and Personalization

Test Data Generator

Artificial Intelligence

See also AI Art on the Software page for more links to do with AI Art Generators.

Address Generator

Written by me as I needed realistic addresses for a work project, this makes realistic looking addresses, but they are randomly generated from a template.

As at the end of movies, I need to state that any address which resembles a real address is purely coincidental.

Buzzword Generator

Sometimes you just need some text to fill up a text field or a document to add to an application, but if you keep using the same text or same document, caching in transport or data retrieval systems could give you a false performance profile. So best to produce text or documents which are actually different.

In addition to the document generation above, which produce HTML pages, these can be converted into PDF and other formats using tooling such as:

Password/UUID Data

Some useful unique text generation.

Time Data

Big Number Generator

My daughter was asking about big numbers, we were looking at Names for large numbers, we found a link for googol but the biggest number on the page, a Millinillion a.k.a. Quingentilliard was not easily found and Katy wanted to see what it looked liked, so I did the following in my Linux shell to show her,
printf "1" ; for A in `seq 1 3003`; do printf 0 ; done ; printf "\n"

General Test Data

Web Design

Fonts

Google provide an amazing collection of free to use fonts at Google Fonts, where you can try out different phrases to see what the typeset looks like. If you like them, you can easily include in your code thus,

In your html page, in the head section put the following syntax (Google even provides there when you click the Select the style option), for example to use the Noto Sans JP font 


    <link rel="preconnect" href="https://fonts.gstatic.com">
    <link href="https://fonts.googleapis.com/css2?family=Noto+Sans+JP:wght@300&display=swap" rel="stylesheet">
OR
    <style>
    @import url('https://fonts.googleapis.com/css2?family=Noto+Sans+JP:wght@300&display=swap');
    </style>

In your css definition


  h1 {
    font-family: 'Noto Sans JP', sans-serif;
  }

See how to use Google Fonts for more info.

Or you could download a TFF or WOFF file and put that in your path and use that as follows (from FontSpace)


    ... in the style section ....
      @font-face {
          font-family: 'TrekFont';
          src: url('font/FinalFrontierOldStyle-8Pg.woff') format('woff'), /* Chrome 6+, Firefox 3.6+, IE 9+, Safari 5.1+ */
               url('font/FinalFrontierOldStyle-8Pg.ttf') format('truetype'); /* Chrome 4+, Firefox 3.5, Opera 10+, Safari 3—5 */
      } 

      .D2 { padding:20px; font-size:36px; font-weight: normal; font-family: TrekFont,    ... etc

    ... then in the body ...
        
        <p class="D2"> Hi </p>

Icons

Character Encoding

Directory List Page Generation

A really super simple way of generating a page listing of a directory tree, using the tree package is:

tree -H . -o site-index.html

Development

Sample Code

Scheduling Execution

The simplest method to schedule execution of some task is using cron, controled via crontab with a schedule, which can be a bit confusing. Take a look at crontab guru which gives you a simple interface to decode or encode a cron entry.

Docker

Docker is an amazing platform to run contained compute resource on, and has become a standard at work. It can be installed on all main platforms including Windows, MacOS and Linux. I tend to use Linux Mint as a Desktop so below I will give an overview of how I installed it on my boxes. The best fit for my Linux distro is Ubuntu so I followed the instructions for this, but I also installed it on a secondary boot Windows install on the same machine and that was also very easy (to be fair, less steps than the Linux install), both installs worked perfectly, but the Linux build ran a bit quicker (on the same machine).

I tend to have dual boot options on my boxes for the kids to play games on, which Windows tends to be better for, though I do play Java Minecraft on Linux (kids play same on Windows).

The summary of the instructions I followed are provided below, yours may vary slightly depending on build and release of Linux 
  
  cat /etc/apt/sources.list.d/additional-repositories.list
  # expect => deb [arch=amd64] https://download.docker.com/linux/ubuntu bionic stable
  cat /etc/apt/sources.list.d/additional-repositories.list
  sudo apt-get update
  sudo apt-get -y  install docker-ce docker-compose
  # options from previous attempt: sudo apt-get install docker-ce-cli containerd.io
  sudo usermod -aG docker $USER
  # this might fail depending on you rights
  docker run --rm -it  --name test alpine:latest /bin/sh
  # then need to run this, which will fail at first, but then will pull the resources automatically, then work
  sudo docker run --rm -it  --name test alpine:latest /bin/sh   
  docker run hello-world

VirtualBox Android

If you want to setup an Android install within a VirtualBox environment see here for steps.
To allow the UI to start (kept getting stuck at the console), display settings had to be set to: VBoxVGA, 16MB, Acceleration Off.

Python

implement-switch-case-statement-python

Java

Java2s Code A great Java coding resources

Git

Git is a great tool, but it does leave behind some junk files, to clean up try:

Shell Script

The shell makes Linux an amazing place to work, I started on Unix at work and though different shells have their own variations, in general everything is transferable (especially if you try to remain POSIX compliant). Here are some great links and tips to help with shell scripting.

Bourne shell (sh) or Bourne Again shell (bash) are amazing environments to work in (along with other shells such as KornShell (ksh) and others), with wonderful tools that can do pretty much anything. Below is a very short list of things I have found useful. They are NOT all my own work, but a collection of Google discovered solutions and some tweaking.

Games

Minecraft

Other

Photography

Astrophotography

Back