Linux

RHEL – Drive Fix (COMPLETE)

April 1, 2023 – 2:35 am

Super excited to say this one would appear to be in the books. I stopped the services and backed up the partition. I didn’t bother trying to shrink it, just blew it away and recreated the home partition and as I write this, xfsrestore is chugging along restoring files. I have already grown the root from 75 GB to 2.1 TB. This seems to have fixed all the issues the web server was exhibiting. Funny how having enough space helps with that.

So just to be clear, the restore takes (in my particular case) 5 times as long as the backup. I don’t think I realized that was the case otherwise I would have probably just picked up a SATA single that I could have internalized instead of the USB 2.0 external drive.

When I rebooted after it was done, I didn’t get to see what happened but I believe it error-checked the drive and rebooted again. My heart skipped a few beats but by that point, I was watching the console and everything was coming up properly.

Initial tests indicate everything is back to the way it was. Email is dropping back into the home/vmail structure. We have oodles of room on the root and home. Life is good today.

Permalink | Comments Off on RHEL – Drive Fix (COMPLETE)
Linux

RHEL – Drive Fix

March 22, 2023 – 12:25 pm

This post is a collection of data related to fixing an issue with the distribution of space coming from a Dell

Services that need to be stopped:

  • postfix.service – Postfix is delivering the mail to home/vmail
  • dovecot.service – Dovecot looks to the home/vmail

Commands that will be run:

XFSDUMP – backs up the /home filesystem to external USB drive:
xfsdump -l 0 -f /mnt/linuxBackup/home.xfsdump /home

xfsrestore -f /mnt/linuxBackup/home.xfsdump /home

Important links

Permalink | Comments Off on RHEL – Drive Fix
Linux

RHEL – Gee Ar Umble…

March 21, 2023 – 5:06 pm

So when I reformatted the server to Red Hat Enterprise Linux, it was basically a fresh start as I got rid of the SAS drives in favor of SATA, there’s a rant someplace on my tech blog about it. Long story short I accepted all the defaults and got on with the business of getting everything that had been set up on Fedora ported over to RHEL. One thing I noticed was mount point / had 75GB while /home had 12TB. Odd but surely there would be a way to shrink home and add it to the root, I’ll contend with it later.

So now it’s later because I have round about 6GB left on the root and now we find XFS filesystems don’t have shrink capabilities… Nice!!! It appears that it wasn’t thought of because hey, storage is cheap, buy and attach new drives and expand the LVM2 VG.

Permalink | Comments Off on RHEL – Gee Ar Umble…
Batch Python

Jupyter Notebook starting directory

February 18, 2023 – 12:20 pm

We love Jupyter notebooks for data manipulation but it’s default starting directory is a bit of a pain on Windows. But you can override this behavior and specify the directory for it to start up with the following command, do make note of the backward slashes, not sure what that is all about but find and replace in normal mode in NPP and you’re golden.

jupyter notebook --notebook-dir="Z:/onedrive_tt/Archive/client/TCC_Touchpoints-Decommissioning/data/outbound"
Permalink | Comments Off on Jupyter Notebook starting directory
Python

Python – Read CSV

December 13, 2022 – 5:16 am

One of the more important things I need to attend to is reading a CSV file and examining it. While there is a plethora of documentation on this, since this is my blog I’m documenting my most used cases.

dfOriginalCSV = pd.read_csv("csvFile.csv", sep=",", dtype=str, keep_default_na=False, encoding='utf-8')

So the file is csvFile.csv, while we don’t have to declare it the sep provides the separator character in case of those pesky pipes. By declaring the dtype of str we’re saying the whole thing is a string so it doesn’t do odd tricks with numbers. The keep default na suppresses pythons overwhelming desire to put nan into anything that doesn’t seem like a proper value and of course always account for the encoding.

Permalink | Comments Off on Python – Read CSV
PowerShell

PowerShell – Count number of commas in each row of a CSV

December 12, 2022 – 7:15 pm

So sometimes your data is so bad that you can’t trust a two column CSV to be turned into a dictionary, or more aptly, the file that is supposed to be two columns won’t open with in python to convert it to a dictionary because it doesn’t see two columns. So where is the issue? Probably a column count, which you can usually figure out if there are extra in excel but what if there are less? This PS snippet will read a given CSV, count the number of columns in the header row and report on any difference in the file, assuming your header row is correct this tells you which lines aren’t helping to validate a file.

$data = Get-Content location.csv

#Count in header
$header = $data[0].Split(",").Count
#Line counter
$i = 1

#Find lines with less or more commas
$data | ForEach-Object { 
    $c = $_.Split(",").Count
    if($c -ne $header) { "Line $i - $c commas" }
    $i++
}
Permalink | Comments Off on PowerShell – Count number of commas in each row of a CSV
Python

Python – FutureWarning

July 26, 2022 – 7:42 pm

Being new to python and running it in the Jupyter notebooks, sometimes you get errors that just don’t make sense and it’s a bit frustrating when you can’t make sense of the error. Let’s take this gem:

/home/aron/anaconda3/lib/python3.9/site-packages/IPython/core/interactiveshell.py:3397: FutureWarning: In a future version of pandas all arguments of read_csv except for the argument 'filepath_or_buffer' will be keyword-only.
  exec(code_obj, self.user_global_ns, self.user_ns)

So what the hell does that mean? I understand pandas, arguments, and what read_csv is but what is keyword-only???

It turns out that this keyword only means that instead of relying on the order of the argument, you simply need to use the keyword for anything except for filepath or buffer…

So this piece of code throws the error:

# Load in the general demographics data.
azdias = pd.read_csv('Udacity_AZDIAS_Subset.csv', ';')

And this is the error fixed:

# Load in the general demographics data.
azdias = pd.read_csv('Udacity_AZDIAS_Subset.csv', delimiter=';')
Permalink | Comments Off on Python – FutureWarning
Python

Python – Dataframe – Unique Value in Column

July 21, 2022 – 9:17 pm

This is the select distinct to get the individual values of a column:

dataFrame.column.unique()
Permalink | Comments Off on Python – Dataframe – Unique Value in Column
Batch

Mute/Unmute System Volume

July 7, 2022 – 2:58 pm

I work from home, so I have an office (room set aside as one) and I try to keep office hours but one of the things that no matter how hard I try, I fail to turn the volume on in the morning so I get notifications for meetings, email and IM’s and turn it off when leaving so I’m not hearing the notification for meetings, email and IM’s. You’d think this is fairly simple, you’d be thinking wrong.
I found a way to pass the pressing of the mute button and was able to schedule it for 8 am and 5 pm. Not elegant because if the mute is on at 5 pm it gets un-muted. It’ll work for now until I can look deeper into it.

Create a batch file that you’ll call with task scheduler. It’s one line:

powershell (new-object -com wscript.shell).SendKeys([char]173)
Permalink | Comments Off on Mute/Unmute System Volume
Education

Command-Line in Current Directory

May 9, 2022 – 8:49 am

My friend and work cohort David Miller keyed me into this last year and it’s been one of the most helpful tips I’ve gotten in years. Need to drop to a command line for the directory you’re currently browsing in Windows explorer? Simply type in cmd in the address bar and you’ve got a command window homed to that directory!!!

Permalink | Comments Off on Command-Line in Current Directory

« Previous Page — Next Page »

« Previous Page — Next Page »