Open Source

Lychee > Piwigo – Transfer

January 1, 2024 – 8:09 pm

So I had been using Piwigo for a while but when I tried to upgrade PHP it blew up the instance. So I started looking around for something else that would support PHP 8+ and drifted into the docker/podman world, and the first container I got working in production happened to be Lychee which is a neat little photo management software but, there is some major lack on Lychee, no user management, no size selection when viewing, no method of putting in a photo description. Then Piwigo version 14 drops and it fixes a lot of the issues that Lychee has and adds some pretty cool functionality so now I’m trying to migrate back to Piwigo, here is the script I’m using to pull out the info for transfer.

--SELECT * FROM photos WHERE album_id = 'qVnjGzkaw25SvFRPek3TpULO'

SELECT 
'move ".\' || replace(sv.short_path, '/', '\') || '" ".\breakout\' || replace(a.title, '/', '-') || '\' || p.title || '.jpg"' as oFileName
FROM size_variants sv
JOIN photos p on (sv.photo_id = p.id)
JOIN base_albums a on (p.album_id = a.id)
WHERE sv.short_path like 'original%'
AND p.album_id != 'qVnjGzkaw25SvFRPek3TpULO'

SELECT 'mkdir "' || replace(a.title, '/', '-') || '"' FROM base_albums

SELECT * 
FROM size_variants sv
JOIN photos p on (sv.photo_id = p.id)
JOIN base_albums a on (p.album_id = a.id)
WHERE sv.short_path like 'original%'
Permalink | Comments Off on Lychee > Piwigo – Transfer
Administration

Domains Transferred

September 22, 2023 – 11:50 pm

So domain registration is just part of life. It’s right up there with death and taxes, well that may be a little exaggerated for standard folk but let’s face it, your domain is an identity. I’m asberry.org the same way I’m Aron Asberry. I have to get my license renewed every so often and it’s the same way with the domains.

In the dark ages companies like GoDaddy preyed upon both novice and experts alike. $2.99 domain registrations opened the door for up-sales and charges up the wazoo. Then Google brought it’s ‘do no evil’ to domain registration with simple yearly pricing, the centralized, intuitive dashboard, and auto privacy as well as included SSL and all was good.

That is until it wasn’t, large companies like Google can’t not be evil forever, just not in their nature. So they sold off the domain registrar bushiness with 10 million domains to Squarespace and clouds began to form and it got dark. SqSp will honor the $12 for a year so the first renewal is at the old price, but then what? I don’t trust SqSp any further than I can throw them.

A ray of hope showed up on my Facebook feel when WordPress.com says they’ll transfer and renew for a year at no charge if you sign up with them. Well why not, I already have an account with WordPress.com and they hook you up with a free year, why not?

So I transferred 7 domains leaving my .us domain behind because it’s not supported. The process was fairly simple and I’m still checking out the DNS and validating that everything is working but so far so good. Not super happy about the interface, not nearly as clean as Google but it does what you need to. There seem to be a lot more add ons like email address and such but nothing too annoying. So far I’m happy but domain registration isn’t a sprint, it’s a marathon and we’ll find out if WordPress is in it for the long run. Here’s hoping!!!

Permalink | Comments Off on Domains Transferred
Batch

Windows CLI – Setting code page with CHCP

April 4, 2023 – 6:11 am

For those times when you are needing to set the command prompt export to UTF-8, it can be done on a temporary basis with ‘chcp 65001’ in the CLI. I’ve been working on achieving parity between the filename in a CSV and on the file system and because the file is rendered in UTF-8, the output of any CLI data needs to be the same so there’s actually a reason that you would want to do this.

Permalink | Comments Off on Windows CLI – Setting code page with CHCP
Linux

RHEL – Drive Fix (COMPLETE)

April 1, 2023 – 2:35 am

Super excited to say this one would appear to be in the books. I stopped the services and backed up the partition. I didn’t bother trying to shrink it, just blew it away and recreated the home partition and as I write this, xfsrestore is chugging along restoring files. I have already grown the root from 75 GB to 2.1 TB. This seems to have fixed all the issues the web server was exhibiting. Funny how having enough space helps with that.

So just to be clear, the restore takes (in my particular case) 5 times as long as the backup. I don’t think I realized that was the case otherwise I would have probably just picked up a SATA single that I could have internalized instead of the USB 2.0 external drive.

When I rebooted after it was done, I didn’t get to see what happened but I believe it error-checked the drive and rebooted again. My heart skipped a few beats but by that point, I was watching the console and everything was coming up properly.

Initial tests indicate everything is back to the way it was. Email is dropping back into the home/vmail structure. We have oodles of room on the root and home. Life is good today.

Permalink | Comments Off on RHEL – Drive Fix (COMPLETE)
Linux

RHEL – Drive Fix

March 22, 2023 – 12:25 pm

This post is a collection of data related to fixing an issue with the distribution of space coming from a Dell

Services that need to be stopped:

  • postfix.service – Postfix is delivering the mail to home/vmail
  • dovecot.service – Dovecot looks to the home/vmail

Commands that will be run:

XFSDUMP – backs up the /home filesystem to external USB drive:
xfsdump -l 0 -f /mnt/linuxBackup/home.xfsdump /home

xfsrestore -f /mnt/linuxBackup/home.xfsdump /home

Important links

Permalink | Comments Off on RHEL – Drive Fix
Linux

RHEL – Gee Ar Umble…

March 21, 2023 – 5:06 pm

So when I reformatted the server to Red Hat Enterprise Linux, it was basically a fresh start as I got rid of the SAS drives in favor of SATA, there’s a rant someplace on my tech blog about it. Long story short I accepted all the defaults and got on with the business of getting everything that had been set up on Fedora ported over to RHEL. One thing I noticed was mount point / had 75GB while /home had 12TB. Odd but surely there would be a way to shrink home and add it to the root, I’ll contend with it later.

So now it’s later because I have round about 6GB left on the root and now we find XFS filesystems don’t have shrink capabilities… Nice!!! It appears that it wasn’t thought of because hey, storage is cheap, buy and attach new drives and expand the LVM2 VG.

Permalink | Comments Off on RHEL – Gee Ar Umble…
Batch Python

Jupyter Notebook starting directory

February 18, 2023 – 12:20 pm

We love Jupyter notebooks for data manipulation but it’s default starting directory is a bit of a pain on Windows. But you can override this behavior and specify the directory for it to start up with the following command, do make note of the backward slashes, not sure what that is all about but find and replace in normal mode in NPP and you’re golden.

jupyter notebook --notebook-dir="Z:/onedrive_tt/Archive/client/TCC_Touchpoints-Decommissioning/data/outbound"
Permalink | Comments Off on Jupyter Notebook starting directory
Python

Python – Read CSV

December 13, 2022 – 5:16 am

One of the more important things I need to attend to is reading a CSV file and examining it. While there is a plethora of documentation on this, since this is my blog I’m documenting my most used cases.

dfOriginalCSV = pd.read_csv("csvFile.csv", sep=",", dtype=str, keep_default_na=False, encoding='utf-8')

So the file is csvFile.csv, while we don’t have to declare it the sep provides the separator character in case of those pesky pipes. By declaring the dtype of str we’re saying the whole thing is a string so it doesn’t do odd tricks with numbers. The keep default na suppresses pythons overwhelming desire to put nan into anything that doesn’t seem like a proper value and of course always account for the encoding.

Permalink | Comments Off on Python – Read CSV
PowerShell

PowerShell – Count number of commas in each row of a CSV

December 12, 2022 – 7:15 pm

So sometimes your data is so bad that you can’t trust a two column CSV to be turned into a dictionary, or more aptly, the file that is supposed to be two columns won’t open with in python to convert it to a dictionary because it doesn’t see two columns. So where is the issue? Probably a column count, which you can usually figure out if there are extra in excel but what if there are less? This PS snippet will read a given CSV, count the number of columns in the header row and report on any difference in the file, assuming your header row is correct this tells you which lines aren’t helping to validate a file.

$data = Get-Content location.csv

#Count in header
$header = $data[0].Split(",").Count
#Line counter
$i = 1

#Find lines with less or more commas
$data | ForEach-Object { 
    $c = $_.Split(",").Count
    if($c -ne $header) { "Line $i - $c commas" }
    $i++
}
Permalink | Comments Off on PowerShell – Count number of commas in each row of a CSV
Python

Python – FutureWarning

July 26, 2022 – 7:42 pm

Being new to python and running it in the Jupyter notebooks, sometimes you get errors that just don’t make sense and it’s a bit frustrating when you can’t make sense of the error. Let’s take this gem:

/home/aron/anaconda3/lib/python3.9/site-packages/IPython/core/interactiveshell.py:3397: FutureWarning: In a future version of pandas all arguments of read_csv except for the argument 'filepath_or_buffer' will be keyword-only.
  exec(code_obj, self.user_global_ns, self.user_ns)

So what the hell does that mean? I understand pandas, arguments, and what read_csv is but what is keyword-only???

It turns out that this keyword only means that instead of relying on the order of the argument, you simply need to use the keyword for anything except for filepath or buffer…

So this piece of code throws the error:

# Load in the general demographics data.
azdias = pd.read_csv('Udacity_AZDIAS_Subset.csv', ';')

And this is the error fixed:

# Load in the general demographics data.
azdias = pd.read_csv('Udacity_AZDIAS_Subset.csv', delimiter=';')
Permalink | Comments Off on Python – FutureWarning

« Previous Page — Next Page »

« Previous Page — Next Page »