Tagged: Linux
‘Linux Command-Line Tips & Tricks’ paperback and ebook updated
Lots of new tips and tricks added. Code snippets are syntax highlighted (only in paperback).
Last week, I moved the last remaining of my books from Ubuntu 10.10 to Linux Mint 20. I changed some covers. I developed a Javascript trick to detect missing images in HTML. I also learned to use ImageMagick
to detect images with CMYK profiles and less than 300 DPI. Book cover images in my backlist had CMYK profiles. These came from the KDP cover template and khtmltopdf
would complain about them. I also wrote two articles for Open Source For You magazine.
- How to tame Mozilla Firefox
This article was about how to lock down Firefox using policies.json file. - BASH Secrets Nobody Talks About
The other article was about some new and important things that I discovered about bash shell programming language.
All of these discoveries were added to my book Linux Command-Line Tips & Tricks. My book-creation process is highly automated. Highly, not fully. The paperback (PDF) creation process is fully automated with shell scripts. My ebook creation process is still partly manual. Calibre has a command-line interface so in future I will fully automate that too.
How to composite videos using FFMPEG
Video overlays are not easy but can be done.
I have two videos and I need to place them side-by-side in a new video. The dimensions first video (blob.mp4) was 1280×720. The second video (aoc.mp4) was 406×720.
The compositing was achieved by using a complex filter:
ffmpeg -i blob.mp4 -i aoc.mp4 -filter_complex "[0:v]pad=1686:720:0:[a]; [a][1:v]overlay=1280:0[b]" -map "[b]" -pix_fmt yuv420p aoc-meets-garbage-disposal.mp4
In the first part of the filter, I specified that the first input video (blob.mp4) will have a dimension of 1686×720, where 1686 is the total width of the both videos side-by-side. A map identifier “[a]” is used for this screen canvas. Next, with a semicolon delimiter, the second video is specified as appearing at 1280 pixels from the left edge at 0. (Yes, it begins at zero. I got a stupid “Overlay area … not within the main area … or zero-sized… Failed to configure input pad on Parsed_overlay_1” error when I thought I had to use 1281 as the starting point of the second video.) The overlaid video is then map identified as “[b]”. This map is then encoded to the output video.
Of course, this video does not have audio. So, I extracted the audio from both videos using FFMPEG, mixed them using Audacity (although I could have done it using FFMPEG as well), and slipstreamed the audio to the composite video using FFMPEG.
ffmpeg -i blob.mp4 -vn blob.mp3 ffmpeg -i aoc.mp4 -vn aoc.mp3 #Created garbage.mp3 from blob.mp3 and aoc.mp3 using Audacity ffmpeg -i aoc-meets-garbage-disposal.mp4 -i garbage.mp3 -codec copy aoc-meets-garbage-disposal-mix.mp4
Alexandria Ocasio-Cortez, the US Congresswoman, lived in New York where garbage disposal machines were banned and people did not buy them even after the ban was lifted. She recently caused worldwide hilarity after posting an ominous video about her discovery of the garbage disposal machine in her sink. She must have at least seen them in movies or TV.
Subhash TweetsToRSS v2019.03.04 for Android, Linux, Mac and Windows
Mostly bug fixes.
- The search form at the top of results page has been revamped and made more meaningful. You can specify the number of results per search.
- The bug which causes the “older tweets” link in the footer to omit the location information has been fixed.
JAR executable (Linux/Mac/Windows) and APK installer (for Android) available at:
http://www.vsubhash.com/subhash-tweetstorss-app-for-android-linux-mac-and-windows.html
EmailTweetor (for Desktop) – the command-line Twitter client released for Linux, Mac and Windows
Like the Android version, it can tweet, reply, delete, favorite and retweet but it cannot alas email.
This terminal-based Twitter client can be operated interactively or automated using command-line parameters. To prevent your bash history from getting cluttered with unsightly tweets, you cannot specify the message in the command line. However, you can do pipe redirection or use a text file. The Twitter client is extremely powerful so I have not documented all of its features in the help.
EmailTweetor was made with Oracle Java version 1.8.0_101. So, ensure that your JVM (Oracle or OpenJDK) is newer or equivalent.
http://www.vsubhash.com/subhash-email-tweetor-app-for-android.html
Compile from source and install Apache HTTP Server 2.4 and PHP 7 on Linux
A how-to guide to build localhost (“virtual host”) PHP 7 sites on the latest Apache HTTP server
I am using a nearly decade-old version of Ubuntu as my main OS. As it runs Gnome 2 and Firestarter, I will be sticking with it until rapture time. However, I did not want the old Apache and PHP from the “Old Releases” Ubuntu repository. Apache HTTP Server 2.4 was released in March this year. I couldn’t care for the new features. The old Apache would have worked just fine but new was new. So, I decided to install the latest Apache HTTP server and PHP 7 by building them from source. There aren’t any tutorials on building Apache 2.4 and PHP 7 from source, particularly from the requirement of a development machine. Besides that many of the configuration files, executables (a2ensite) and directories seem to be missing after the compilation/installation process was complete. It required a bit of thinking and RTFM. So, I decided to put it all down here in one place.
Apache HTTP Server 2.4 (httpd) compilation, installation and configuration
First, download the source files.
- Apache HTTP Server (httpd) – https://httpd.apache.org/. Check the documentation at https://httpd.apache.org/docs/2.4/new_features_2_4.html to see if you need anything more than what is given here. I am not install things like SSL as my requirement is to build and test localhost (virtual host) PHP 7 websites on my laptop.
- Perl Compatible Regular Expressions (PCRE) – https://pcre.org/
- Apache Runtime – https://apr.apache.org/
cd ~/Build wget -c url-of-pcre-archive wget -c url-of-apache-httpd-archive wget -c url-of-apr-util wget -c url-of-apr
Extract the source files of PCRE and httpd in a directory named Build. The source files of the Apache Runtime need to be extracted inside the srclib directory of the httpd directory.
cd ~/Build tar -xf pcre* tar -xjf http*.bz* cd http* cd srclib tar -xf ../../apr-1* tar -xf ../../apr-util-1* ln -s apr-1* apr ln -s apr-util-1* apr-util
Next, compile and install PCRE to some directory (/usr/local/pcre).
cd ~/Build cd pcre* ./configure --prefix=/usr/local/pcre make sudo make install # Installs PCRE to directory /usr/local/pcre
Finally, compile Apache httpd to some directory (/opt/httpd) with specified PCRE and Apache Runtime directories.
cd ~/Build cd http* ./configure --prefix=/opt/httpd --with-included-apr --with-pcre=/usr/local/pcre make sudo make install # Install Apache httpd to directory /opt/httpd
The Apache HTTP Server has been installed. Now, it is time to start it and check it in a browser.
sudo /opt/httpd/bin/apachectl -k start # Start server firefox http://localhost # Serves /opt/httpd/htdocs/index.html
The configuration files of Apache httpd 2.4 seem to be different from the ones from the Apache installed from the old Ubuntu repositories. The main configuration file will be at /opt/httpd/conf/httpd.conf
PHP7 compilation, installation and configuration
Next, compile and install PHP 7 to some directory (/opt/php-7). When executing the configure statement, you need to specify the location of the Apache Extension Tool (/opt/httpd/bin/apxs).
sudo /opt/httpd/bin/apachectl -k stop # Stop Apache HTTP Server cd ~/Build cd php-7* ./configure --help | more read -p "Press enter to continue" oNothing ./configure -with-apxs2=/opt/httpd/bin/apxs --prefix=/opt/php/php-7/arch-all --exec-prefix=/opt/php/php-7/arch make make test # This step is optional sudo make install # Install PHP to directory /opt/php/php-7 sudo libtool --finish ~/Build/php-7/libs # Adds PHP 7 module loader to httpd.conf
This completes the PHP 7 installation. Please note that the libtool statement writes the following line to Apache server’s httpd.conf
file.
LoadModule php7_module modules/libphp7.so
After PHP 7 has been installed, add a PHP file extension handler to Apache server’s httpd.conf
file.
# The following is a multi-line statement. Copy all of them. echo ' <FilesMatch \.php>; SetHandler application/x-httpd-php </FilesMatch>' | sudo tee -a /opt/httpd/conf/httpd.conf
Copy the development php.ini
in the Build
directory to /opt/php/php-7/arch/lib
directory.
cd ~/Build/php-7* sudo cp php.ini-development /opt/php/php-7/arch/lib/php.ini
Modify this file appropriately for your developmental setup. For production, you will have to use modify and use php.ini-production as the php.ini file. This completes the PHP installation and configuration.
Now, it is time to test .it. Create a sample PHP file in the Apache server’s htdocs directory and request it in a browser.
echo '<? phpinfo(); ?>' > php-test.php sudo mv ./php-test.php /opt/httpd/htdocs/php-test.php sudo /opt/httpd/bin/apachectl -k start # Start Apache HTTP Server firefox http://localhost/php-test.php
This will prove that your HTTPd server can handle PHP. However, to develop websites on your local development machine, some more changes need to be made. You can delete the /opt/httpd/htdocs/php-test.php
file.
Apache HTTP Server reconfiguration for virtual hosts
First, append a line in your /etc/hosts
file so that browser requests to your test site (say me-website.com) are routed to your local installation of the httpd server.
echo "127.0.0.1 me-website.com" | sudo tee -a /etc/hosts
Next, clean up httpd.conf by removing the PHP file handler and the PHP loadmodule statement. Instead, append the following lines to httpd.conf.
# Added by me Include conf/me-websites.conf
Now, httpd.conf is mostly in its original state, save for the line that tells httpd to load the new me-websites.conf configuration file. Then, create the file me-websites.conf
file and put all your virtual hosts configurations in it. Copy this file to /opt/httpd/conf
directory. This will ensure that PHP scripting is limited to your “virtual host” sites.
Listen 8080 <VirtualHost *:8080> ServerAdmin webmaster@me-website.com DocumentRoot "/home/account-name/Websites/PHP/me-website/htdocs" ServerName me-website.com ServerAlias www.me-website.com ErrorLog "/home/account-name/Websites/PHP/me-website/logs/me-website.com.error.log" TransferLog "/home/account-name/Websites/PHP/me-website/logs/me-website.com.access.log" LoadModule rewrite_module modules/mod_rewrite.so LoadModule php7_module modules/libphp7.so <FilesMatch \.php$> SetHandler application/x-httpd-php </FilesMatch> <Directory "/home/account-name/Websites/PHP/me-website/htdocs"> Options Indexes FollowSymLinks AllowOverride All Require all granted </Directory> </VirtualHost>
Next, add your account to the user group www-data
. Your account can be the owner of the directory /home/account-name/Websites/PHP/me-website/htdocs
but you need to give read, write and execute permissions to www-data
. Now, you can put your PHP scripts in /home/account-name/Websites/PHP/me-website/htdocs and start coding. You can test the site in a browser at http://me-website.com:8080/
If you would like to test it on port 80, then remove the “Listen 80” in httpd.conf and replace 8080 in inside the VirtualHost directive in me-websites.conf
. Then, you can simply test your sites at http://me-website.com/ or http://me-website.com:80.
Fix for autogenerate bug in Eclipse PHP run configuration
I have already published how to modify the Eclipse source files to fix the autogenerate bug in the Run configuration of Eclipse PHP. If that is too much for you, then you can simply let Apache HTTP Server’s rewrite module fix it for you. Create a .htaccess file in your home folder. This will remove the your Eclipse PHP project name for the run configuration.
RewriteEngine On Redirect "/me-website" "/"
Interestingly, the propriety IDE (derived from the same open-source project) sold by the creator of PHP does not suffer from this PDT “bug”.
Create a Twitter Archive – A PDF dump of all tweets from an account
I have 51 tweets in my @SubhashBrowser account. So, this command will create the Twitter dump.
bash twitter-to-pdf.txt SubhashBrowser 51
Subhash TweetsToRSS must be running when you execute this command. You also need wkhtmltopdf
and pdftk
installed. The content of twitter-to-pdf.txt is as follows:
# Initialize variables iPage=1 iMax=1 iRemainder=0 sDocs="" let "iMax = $2 / 25" let "iRemainder = $2 % 25" if [ $iRemainder -gt 0 ]; then let "iMax = iMax + 1" fi sJavaScript = "(function(){ " + " var arIframes=document.getElementsByTagName('iframe');" + " var n = arIframes.length; " + " for (var i=0;i<n;i++) { " + " arIframes[0].parentElement.removeChild(arIframes[0]); " + "}})()" # Javascript to remove YouTube iframes generated by TweetsToRSS # Create archives for (( iPage=1; iPage<=iMax; iPage++ )) do echo "Converting http://localhost:8080/?q=%40$1&output=html&older-than=$iPage" wkhtmltopdf --quiet \ --encoding utf-8 \ --debug-javascript --javascript-delay 60 --run-script "$sJavaScript" \ --user-style-sheet wkhtml_twitter_style.css \ --image-dpi 300 -s A3 \ --margin-top 0 --margin-right 0 --margin-bottom 0 --margin-left 0 \ "http://localhost:8080/?q=%40$1&output=html&older-than=$iPage" \ "@$1-archive-$iPage.pdf" sDocs="$sDocs @$1-archive-$iPage.pdf" done # Combine archives pdftk $sDocs cat output Complete-Twitter-Archive-of-@$1.pdf # Updated metadata echo "InfoKey: Creator" > $1-meta.txt echo "InfoValue: Subhash TweetsToRSS (www.vsubhash.com)" >> $1-meta.txt echo "InfoKey: Title" >> $1-meta.txt echo "InfoValue: The complete Twitter archive of @$1" >> $1-meta.txt echo "InfoKey: Subject" >> $1-meta.txt echo "InfoValue: Collection of all tweets from @$1" >> $1-meta.txt echo "InfoKey: Author" >> $1-meta.txt echo "InfoValue: V. Subhash" >> $1-meta.txt echo "InfoKey: Keywords" >> $1-meta.txt echo "InfoValue: $1, twitter, tweets, archive" >> $1-meta.txt pdftk Complete-Twitter-Archive-of-@$1.pdf \ update_info $1-meta.txt \ output The-Complete-Twitter-Archive-of-@$1.pdf # Cleanup mkdir $1-backup if [ -d $1-backup ]; then mv @$1-archive-*.pdf ./$1-backup mv Complete-Twitter-Archive-of-@$1.pdf ./$1-backup mv $1-meta.txt ./$1-backup fi
Twitter seems to have placed a limit of just above 3000 to the number of tweets you can pull from an account. Here is the complete Twitter archive of US President Donald J Trump (@realDonaldTrump).
Cool Terminal Tricks – new article in OSFY mag August 2016 edition
In the stands.
How to run Android without Google!
And “Cool FFMPEG tricks” – two articles in this month’s Open Source For You (OSFY) magazine.
Nimbus theme DEB files for Gnome 2 and Mate desktops
Install the icons DEB file first.
Theme: https://www.dropbox.com/sh/4ln23fhy30zydro/AACXpaEg7hkLPhSF7jNEPEE3a/Other-Free-Software/gtk2-engines-nimbus_0.1.4-2_amd64.deb?dl=0
Icons: https://www.dropbox.com/sh/4ln23fhy30zydro/AAAI4NZ08geO0nz09kx0lCXQa/Other-Free-Software/nimbus-icon-theme_0.1.4-2_all.deb?dl=0
Motorala Motodev Studio
https://www.dropbox.com/sh/4ln23fhy30zydro/AACqmFy1RjCbg462g8_0e725a/Other-Free-Software?dl=0