fyr.io - Everything (full)

Backblaze Account Takeover

2026-04-15

Backblaze Account Takeover

Posted on Wednesday 15th of April 2026 at 23:59

Last updated on Thursday 16th of April 2026 at 09:42

Sometimes, you find a way to abuse a system and you don't get a bounty payout. Such is life! This is what happened here, but... It is a bit different to the usual tale from the world of infosec, because I didn't technically exploit anything, yet still managed to perform account takeovers on well known backup provider Backblaze, including "accidentally" taking over a legitimate customer account (which contained live, real world data). I merely abused a somewhat poorly constructed sentence in the official documentation (which happened to contain a compliant and unregistered domain.) I... Guess it's exploiting the interpretation of the English language? Well, sort of. This is pretty much just your standard registering an unregistered domain that's referenced somewhere but thought the journey was somewhat interesting, so here it is.

Backblaze won't pay out a bounty if you publicly disclose, but as this is seemingly not worth a payout anyway and has been fixed for a few months now, what's the harm in publishing, eh?

Back in April & May 2025 I was trialing Backblaze for a data project at work and it went really well. The costs were low, the tech worked perfectly. It's a great service & product, and I knew it was the right tool for the job. I had mucked around with it a fair amount, using an account associated to my work email address, but I wanted to start a fresh account for the Production version of the data project.

Thankfully, Backblaze has a documented method (<- archive.org link) to change the email address associated to your account. Line 4 under the aforementioned "Remove an Email Address from a Previous Account or Trial" is the important one:

Enter your current password, and enter and confirm a new email address that looks like this: xxx8xx8@x87xx.xxx.

Upon reading this, my initial interpretation was to recoil, eyebrow arched, and question it - it wants me to change my accounts' email address to xx87xx8x@x87xx.xxx specifically? That's... Odd, right?

I pondered it a bit...

Maybe they have some logic in the process that picks up that exact string of characters and black-holes the account or treats it differently? In fact, that seems like a strange and oddly specific string to use, why wouldn't they just use a non-RFC5322 string instead of a valid-looking email address and pluck it out prior to the validation rule triggering? Wait... What if they don't do any of that and they've just poorly worded that line in the docs?

At this point, I reconsidered the line in the documentation. I could also interpret it to mean "change the email address to something like this but not this exactly".

But if I - a native English speaker - had interpreted it on first read as it wanting me to use that funky x-laden email address specifically, others would too. So I did the only thing I could do - I hopped onto a registrar and checked to see if the domain used in the specified email address was available (as is habit for many infosec oriented people I'm sure) and to my surprise, it was available!

Now, I took a moment here. This wasn't a boring old .com - it is a .xxx domain. They're not the cheapest domains to rent! To be fair they're also not overly expensive either, if you're not trying to register a premium domain - a common word prior to the .tld - which xx87xx isn't of course... but money is kinda tight. I considered whether I wanted to pursue this for a while, a few weeks at least.

Eventually I decided to use a chunk of money from my personal monthly "treats" pot and go ahead and register it. A sacrifice of a fewer shop-bought drinks and treats for a while, on a hunch. But a fun hunch!

A primer on .xxx domains

.xxx domains, launched in 2011, are for... well, porn stuff. If you host a porn site, you don't need to use a .xxx (or similar) domain but it is a recommendation, to ease filtering pretty much. Obviously it's pretty useless if it's not a hard requirement, but when there's money to be made who cares about logic?

Unbeknownst to me at the time of forking over £60, .xxx domains are controlled by ICMRegistry, which appears to quietly be... GoDaddy. You can register the domain itself easily enough, via your nearest high quality registrar, but in order to actually configure records for one of these domains, you need to evidence that you're a member of the pornography business.

Yep.

So I had to send proof of my participation in the porn industry to GoDaddy via ICMRegistry.biz, a totally not sus domain. Once they've verified you are in fact a porn star (or otherwise work in the industry) you'll get a membership token, which can be used in the registrar to unlock the domain, at which point you can use it like any other domain (as long as you follow their rules of course.)

So it is clear to any readers here: no, I am not a participant in the porn industry. I've never been in it, directed it, done CGI, fluffing, fetching drinks, cleaning... But we won't let that stop us, will we?

I registered, because of course I did, and submitted a bare-bones application. But I heard nothing back.

Those of you who have been involved in web stuff for a while will know that GoDaddy are renown for being ...let's charitably say... disorganised. So after waiting a bit to hear back, I started pestering. Eventually, someone must've gotten fed up of me and let my application slip through, as one random evening I got confirmation emails and my membership token without any feedback on my pleas for help. So I guess as of 22nd May 2025 my membership of the porn industry is verfied and as a result I find myself with a working .xxx domain! Yay?

So I've now got a working .xxx domain, which I of course immediately hooked up to my Proton email account so it can receive emails sent to my shiny new x87xx.xxx domain, and began on the next phase of the plan!

After checking that the documentation hadn't been updated and still said the same thing (it did) I decided to test the process by creating an account and going through the steps as documented. I opened Backblaze, logged into my account, initiated the account email address transfer process (sending the account over to "xx87xx8x@x87xx.xxx"), and... Blocked. Account already in use.

Crap.

No, wait, not just 'crap' - Holy crap!

(This gets a little convoluted. Hopefully it can be followed along easily!)

I went back to the backblaze login page and went through the 'forgotten password' process, entering the email address from the documentation (again, xx87xx8x@x87xx.xxx) and... Ping! I get an email from Backblaze on my freshly minted email alias. Within this email is a link to reset my account password!

I follow the password reset process and can then log in. At some point in the past, someone has indeed already followed this document, (mis)interpreted the process and transferred the account to this email address.

However... This isn't the legitimate customer account I mentioned earlier. That comes later.

I'm buzzing. I'm logged into someone's account - it's empty of all data and details, but it's there. I need to test this properly, so I go back through the account email address transfer process and change this newly acquired account email address to [somethingRandom]@x87xx.xxx - that way I can retain access to the account (as I was using a catch-all on Proton - [anythingHere]@x87xx.xxx will come into my mailbox) and free up the email address to try transferring another account which is under my control to determine what the procedure is exactly.

Next, I log out, then back in to my original test account from earlier and try to transfer it again. It flies through the process. No email confirmation required, it just changes the owner of the account straight over to the .xxx email address.

I run a few tests on some other accounts, write this up as a "I don't know how to classify this but I can kinda take over customer accounts if they follow your documentation?" bug report on the bugcrowd for Backblaze and feel great that my hunch paid off.

Bugcrowd close the issue.

The first reason it's closed is because this issue relies on an attacker purchasing a domain, as if that's onerous for an attacker to do. Also, it's not in scope, they say, as it's on the documentation site, not the base domain. I clarify that the issue isn't on the documentation site but as a result of the content on it and ask for the documentation to be updated anyway, but nope. The triage process has blocked this and -1'd me to boot. Bugcrowd has shot me down.

Ah well. I know it's legit. It works. So what better way to prove it than to wait for a customer to follow this guide and send me their account?

I'm tapping away at my keyboard at work in January and my phone dings - my Backblaze account has been transferred.

After a couple of underwear-browning seconds my eyes widen in realisation - Hold up, it's an email that landed in my .xxx domains email inbox! I take a closer look and, sure enough, some [randomUser]@gmail.com has seemingly followed the documentation and transferred their account to xx87xx8x@x87xx.xxx!

I finish the days work, head home and get online. Step one, account password reset, which works, so I log in to it. Sure enough, someone has initiated this to the documented email address and as a result has given me access to their Backblaze account. This legit, paying customer account includes two computer backups with current, very recent data. It's live data, uploaded moments ago.

Now, that bit right here that I've just done? Morally questionable at best. I've just obtained access to an active, paying customer account, and I've taken things a step further than is generally acceptable to me and the community by resetting their account password and verifying it is accessible.

As their computer still backs up to this account, I immediately get another bugcrowd ticket logged with a plea to reconsider the validity of this as... not a bug per se, but a problem for sure. I don't touch the account beyond my initial logging in and seeing the data shown on the dashboard, I don't download anything or navigate anywhere in the interface. I log out after taking a screenshot, and fire off my plea. I know, trust me, bro, but I really didn't. Backblaze will log all of that.

Backblaze helpfully includes the accounts originating email address in the transfer notification email, so I send the owner an email from my personal (fyr.io) email account clarifying the situation. Their account has come to me, I won't touch it, I've emailed Backblaze and am trying to get them to sort it. I don't hear back.

I did strongly consider transferring the account back via the same process, and perhaps on reflection I should have done this, however I didn't want to touch anything on the account at all so opted to let Bugcrowd and Backblaze handle it themselves. They (Bugcrowd) have been notified that I have someone else's account now, so they'll be interested in getting this resolved quickly.

Or not? Bugcrowd strikes again.

They reject this second attempt at communicating a problem. I have been advised to email Backblaze' documentation provider, Document360, as the issue is, they claim, on the documentation site (it isn't) so I go through the motions. They quickly reply, of course stating that the issue isn't on the documentation site (see, it isn't!) and to email Backblaze directly as they're the ones that write the documentation.

So I go looking for someone to poke about this account transfer at Backblaze directly. Yes it may not be a typical hack or an exploit but it is a real problem. I just want the docs to be fixed, darn it! I find their web based chat and consider my efforts to be coming to a conclusion now that I get to explain to an actual human over chat wha-oh, wait, nevermind. I forgot it's the age of slop. Instead of a human I have to chat to their LLM bot (yay.) which asks me to email Backblaze's 'report phishing' email account. Seems... irrelevant, but okay, I guess I can see how it might help and it actually has a chance of crossing over to the right person. I email them. They reply saying it's nothing to do with them, and to go through their bug reporting page (bugcrowd) so that's a dead end right there. Whilst waiting on the phishing email address reply, I've logged a support ticket on the Backblaze Zendesk portal.

Whilst waiting for a response I have also written another update on the bugcrowd ticket. Which, of course, sits ignored.

Then, I hear back from a support person. Not just any support person, but a support person who actually takes the time to look into the issue. I believe they understood what I was getting at after a bit of back and forth and escalated the issue, as the next day the ticket on bugcrowd has a burst of activity, it's rated a p4, then a p5 straight after (informational, therefore not deemed worthy of a reward, booooo!) and then marked as resolved.

The documentation page (<- the actual documentation page) is updated, removing the whole section containing the .xxx email address.

13 days after the bug is marked resolved, the account that was accidentally transferred to me in January is finally transferred away (I assume back to the original owner?) and I can relax now I don't have potential access to this strangers data anymore.

So in the end I'm left with yet another bad bugcrowd experience, a .xxx domain (which I guess I could claim proves I'm in the porno industry now?) and I'm £60 worse off for it. But hey, the issue is fixed and someone has their data back, so in the end a net positive.

What am I gonna do with a nonsense .xxx domain?

Cat Tax

2026-04-12

Cat Tax

Posted on Sunday 12th of April 2026 at 22:48

We got two cats about a month ago, so I'm in debt cat-tax-wise - here's payment, I hope it is sufficient.

These two are siblings, brother and sister, and have been in a cat rescue place since birth pretty much. They've had 8 months of infections, ailments and other issues (including one close call with an eye infection, leaving a permanent mark) which has delayed their adoption by quite a while. Their siblings all left the cattery a while ago so they've been left waiting all this time alone. Together, but alone. They've become best friends whilst in there and are absolutely a team of two.

Despite that, they're super friendly and affectionate to humans. It didn't take long for them to settle in at all and take over. They've definitely become part of the family.

We've had two cats before, but at different times, not together. There's something kinda cozy about having two cats that get along so well with each other.

That puts the pet count at seven - two cats, two guinea pigs and three chickens!

Dandelion Honey

2026-04-12

Dandelion Honey

Posted on Sunday 12th of April 2026 at 00:11

Last updated on Sunday 12th of April 2026 at 22:49

We're a little late to start the garden stuff this year but we've mostly prepped our planters and begun to plant the seeds that will grow into food we'll eat daily in a few months time. Side note: I need to write about this part of my life more, I need to port over my old wordpress posts about it, and also build category/tag filter functionality into the site... the data is there, I just gotta do the logic bits and the copy/paste bits!!

Each year we edge slightly more self-sufficient than before by growing more and figure out what works for us and what doesn't. As we look forward to growing more of our own food again this year, we find ourselves learning a bunch of cool stuff. My wife was reading things and learned one such cool stuff: by mixing dandelion petals, water and sugar you can make a honey-like substance! We're not lucky enough to have 5 gallons of maple syrup on the go but this'll do us for now! :D

Dandelions are everywhere - they're a bit of a super food in that the entire plant is edible and contains a bunch of nutrients and as a bonus they grow pretty much everywhere. Odds are you'll be able to find them near where you are (unless you're in Antarctica.) I've heard before that they grow where soil is unhealthy and in fact help restore its health, which is pretty great if true. Wikipedia states It is kept as a companion plant; its taproot brings up nutrients for shallow-rooting plants. It is also known to attract pollinating insects and release ethylene gas, which helps fruit to ripen. so I'll continue to assume that to be true and am now considering cultivating it in the greenhouse...

After sharing this with us, my son and I went out around the house and nearby (safe and clean) areas and picked about 4 cups-worth of the yellow flower, leaving the leaves and stem in place. Upon returning home my wife took over from the kid and we both pulled the yellow petals from the flower heads, dropping them straight into a saucepan, trying to minimise the amount of green stuff that ended up in there as it seemingly becomes bitter if you leave too much in there. We didn't wash them, as this would wash away any remaining nectar.

The remains of the flowers have gone in a container for our two guinea pigs.

The saucepan was filled with around 250ml of water and gently boiled. After straining, the yellow liquid was returned to the saucepan and 250g(!) of sugar was added (I know right. We've been told to match the volume of water in ml with the weight of sugar in grams so if you use more or less water, you might want to adjust the sugar number) We then stirred this a bunch as it heated back up gently, then let this simmer on a low heat for about an hour and a half to reduce it down. After cooling a little in the saucepan the 'honey' went into a jar to let it cool to room temperature, at which point it went into the fridge.

The resulting honey is way thinner than your regular bee-origin honey, but it tastes very similar. Great for adding to tea, toast, or our current breakfast favourite, overnight chia seed pudding.

This was our first batch, a small test to see how it tasted (it's pretty great) but we already have some changes we'd make:

Pick them on a warm, sunny day: They'll apparently have more nectar in there, opening up more to attract pollinators. We picked these in a fairly mild (approx 12 celcius) temperature, where it had been overcast for much of the day
Pick them in the morning: We picked these in the evening. Any nectar would have been slurped up by insects already.
Pick way more: we want way more of this. If we could cook a big batch that'd be great. It's so much sugar though but honey is also very high in sugar!
Use less water: we're thinking less water will result in a thicker substance! Experiments will determine if this pans out!

If it's warm tomorrow I'll probably go pick some more!

RSS Feed Changes

2026-03-18

RSS Feed Changes

Posted on Wednesday 18th of March 2026 at 23:50

Hello, you lovely little RSS feed reader you! RSS feeds are pretty cool, aren't they? I recently got a new phone but am yet to sort out a feed reader on it. I still use my old phone, so I'm not missing out! But I wanted to get two things added to the site before I begun the search of the perfect mobile reader (or signed up to one of the homebrew web based ones a couple people I know are maintaining):

Add the ability to post "secret" RSS-only articles (like this one!)
Add a "full" feed for all three of my existing RSS feeds which includes the entire contents of the post/scrap

I've now done these things on the big tidy up task I've just been through. I saw the "post hidden content just for your RSS feed users" idea on Mastodon months ago and have wanted to do it ever since. Now I can! :D These secret posts that show up on RSS but not on the list of posts (or scraps) are still viewable in a web browser if you know the URL! As for the full RSS feeds, if you're so inclined, feel free to adjust your RSS feed subscription to one of the following RSS feeds instead of the one you're currently subscribed to! They are:

The "everything" feed, the "posts" feed, and the "scraps" feed, which each come in two flavours: "Title & Summary only" (which is the original feed type), or "Title, Summary & full contents" (the new one!)

You can find these feed links on the /feeds page!

I wrangle this site myself, it's home grown with minimal third party libraries (none on your end, only a couple built-in PHP ones!) and... well hey, let's just say I ain't a coder, alright? I'm a scrappy slinger of PHP, HTML & CSS, so I'm bound to have made mistakes. If you notice any errors or wish to suggest improvements in both the code, the output, or within article content itself, lemme know! And speaking of making mistakes...:

Oh one more thing

Ages ago I made some changes here and there and all of a sudden, all the posts in the fyr.io feed came back as unread! I didn't know what caused it at the time, but now I do. For some reason I'm using YYYY-MM-DD for the <pubDate>, whereas before I was using the CORRECT method, which is to use an RFC822 formatted time & date! Doh! Sorry about this if you saw it. I want to fix it, but I also want to not piss off you, my wonderful RSS feed readers (again) so... maybe I'll post another RSS only post alerting people to the change well in advance.

That's all, have a good one!

Absent

2026-03-14

Absent

Posted on Saturday 14th of March 2026 at 00:29

I haven't published here in a while, nor made much in the way of changes. I've got a post or three living in purgatory - a long overdue scraps update (thank any and all dieties that Scrolls has returned with vigor), a failed but-still-interesting bug bounty to write up, and a secondary continuation of my quick research into the (lack of) quality of school email setups - and a few others drafted or planned out, but honestly? I've been too busy. I don't often get overly personal here, but heck let's get all human up in this web and give it a go.

Shellsharks recently wrote an amazing piece on burnout - I've been there, brother - and it ~~inspired~~ ~~pushed~~ motivated me to expend some effort here on this post, as I've missed this site dearly. I am not suffering from active burnout right now, but I can feel the creeping edge of it setting in. Recognising it is half the battle, and I'm expecting that doing the web equivalent of touching grass (read: poking about in HTML land) will help realign myself somewhat, giving me a little extra ammo to fight it off.

But why am I recognising the early signs of burnout? It's several things, as is often the case! Allow me to go through the reasons (whilst listening to one of my favourite bands to ever grace my ear holes) mostly for my own benefit!

Late last year I became aware of significant organisational changes that were going to occur at my workplace - changes that are the equivalent of our organisation being bought out. The reasons for this needing to happen are understandable yet frustrating, as to me they could have been avoidable. However I'm basing that off of limited information, and the reality of the situation is never as clean and clear cut as it ought to be. With everything happening around that at the time, the worry and concern about the uncertainty of the future added a very particular pressure on to all of us there.

Following this, a more immediate financial issue reared its ugly head, and presented in the only way it can in the situation we all found ourselves in: redundancies. It became clear to me that something was afoot, and that quickly evolved into the knowledge that, yes, there would be redundancies in the future. This was a difficult time as we were all working hard to keep everything moving forward with the buyout. We didn't back away, throw our hands in the air or give up. We knuckled down. We committed. We guaranteed that this would be the smoothest buyout they have or would ever experience (at least, from a technology perspective!) We provided data, we realigned our objectives, and we were actually motivated to do this. We committed what little budget we had to certain technology and software solutions at their request (though it was more like a demand, but okay, we want to help both orgs) and hit the ground running preparing the environments we maintain for the transition. During this, the redundancies began, and we have so far, at the time of writing, not had any in the department. I don't think this is because we're doing this work, but because we're literally running a skeleton crew already, and have been for half a decade.

The concern was still ever present and at the forefront of my mind throughout this time. Accommodating a very likely and very different future whilst also trying to keep on top of everything else was certainly a challenge. Doing it all whilst having some pretty severe financial anxiety plagueing me literally 24/7 since December - a viciously expensive month in any year - was not a walk in the park. I have always had an overbearing financial anxiety, it comes in waves, and these last few months I've been submerged. Because it's a frequent occurence for me, I have some methods of managing it. I maintain a (way over-engineered) budget spreadsheet and track everything, I try to keep a healthy emergency-fund, I read and play videogames to both distract and relax, and, most importantly, I have access to the worlds most perfect humans in the form of my wife and my son. She is my rock, my safe place, and my shoulder-shaker, bringing me back to the present instead of living in the worst future I can imagine. I adore her. And my son? He is everything.

Now we get to last week. What a blur these last two weeks have been. At the start of the week, our boss (we're a team of four - me, doing sysadmin stuff, two techies, and the boss, who does the managing, project planning and so on) handed in his notice. He resigned, effective Friday that same week. There's no grievance there - in his position I would have likely done the same. An opportunity came along that he had wanted to do for a while and therefore couldn't say no to, and he took it. I respect him an equal amount now as I did back then and before this happened. There's no hard feelings, we're still in contact, we play videogames socially, it's all good and he made the right call...

...but wow. These things have ways of happening at the worst time. Not only are we about to begin the biggest data and platform migration we have ever done (I literally started getting the migration tools configured on his last day, ready for this coming week) but we're doing the same migration across six different organisations all at the same time. So, I'm doing my solo sysadmin across six orgs, all of my bosses project planning and day to day work, budgeting, all of the migrations and acting as the escalation point for issues the two techs can't solve. Oh, speaking of the two techs, they're worth their weight in gold. They stepped up before with the news of this buyout, but they've somehow found the energy to cover so much of my stuff this week. I want to promote them, pay them more, reward them somehow. But given... everything... I can't see anything like that happening any time soon. I will buy them food, I will be overwhelmingly vocal when I praise them, and will do anything I can to support them. I would be drowning if not for them.

So that was the situation last Friday. This Monday, we found out the org doing the buyout have changed their minds at the 11th hour. They're not going ahead. Why? Only they know. But this decision has left us mid-transition with zero support. What this means for... well, everything, is anyones guess. All we know is the massive negative emotional impact it's had on everyone involved on our end. We're reeling, and will be for a long time. The consequences of this will be felt for years to come. The migrations must continue as planned, as contracts have been signed, so there's no less work. Just way more uncertainty.

So this week I have been stretched, pulled, squashed, pushed, thrown and trampled, all after three months of steadily building pressure and uncertainty. It's a lot. I'm okay though. I've learned a lot since my last bout of burnout, and I'm managing myself really well, all things considered. The financial anxiety is back, the stress, the worry and concern for colleagues and friends. Yeah. It's a lot. I'm dropping what can be dropped, I'm delegating what doesn't need me (hopefully sensibly, without adding too much pressure on others!) and triaging the stuff I do need to do as best I can. It helps. What also helps is seeing that all this stuff we're trying to accomplish... is actually working! We've had bumps in the road, but you know what? I'm gonna go ahead and say it: We do good work. We're good at this stuff. We're pulling it off. It's good to take a look at the accomplishments and successes and remind ourselves of how far things have come despite it all.

Throughout all this, there has been some positive stuff!

You may have noticed, if you have been here before, that I have a soft spot for the 32-bit cafe - a community of web weavers, bloggers, digital gardeners and sharers of joy and knowledge. Back in December I made an off-hand comment about helping them out a bit, and xandra asked if I wanted to help out with their infrastructure in an official capacity! I was, and am still to this day, both honoured and bemused (because who am I but some lacky imposter, right?!) and have become even closer to the team of brilliant barista's that make the 32-bit cafe what it is - a kind, friendly community of people who care about the web and each other. It has legitimately become my third place.

I have plans for the 32-bit cafe. Many plans, many ideas. But with everything going on in my workplace I've found myself getting frustrated that I can't commit enough time and resources to this community as I had originally planned and hoped for. There has been some progress though - I have set up a Jitsi server for the webweaving workshops and have begun documenting their infra as it stands (for myself as a way to learn it all) but my input has been frustratingly limited thus far. <gallows-humour>But hey, if I am made redundant in the future I'll have way more time to commit to the cafe!!</gallows-humour>

Guild Wars 1 has had a bit of a revival with the Reforged updates. Over the years since playing it back in 2007 I have found myself dropping into pre-searing to see what's going on, get cozy and nostalgic before heading elsewhere in life, and this rebranded patching thing they're doing has generated interest in a few people I know (one of which is the boss I mentioned above.) As a result, we now have a guild wars playing session every Tuesday evening. It's only for an hour or two, enough to chip away at a few quests, but it's really nice. We're planning on doing the Prophecies campaign at least, hitting the missions when we're all online or side quests when one or two of us are unavailable.

This weekend just gone, we increased our pet count by two by adopting two amazing cats from a rescue place about an hour away from here. I owe cat tax, yes yes I know. I'll get to it! They're the cutest little guys I've ever had the pleasure of knowing and have settled in to our home incredibly quickly.

The final thing I want to talk about, the final positive thing, is something that is sweeping particular group of brown coat wearing people by storm at the moment. When describing the above negative things to someone earlier today, I said something like all this shit stuff that's going on is like... it's like I'm a campfire, and I've had a bucket of water thrown over me - the flames are being exstinguished. But... swipe away at the wet ashes and there's a glowing ember in there. That glowing ember? It's whatever the hell is happening with Firefly right now! I love Firefly, and Nathan Fillion is hinting at something happening. To be announced this Sunday. I haven't been hyped for anything for years.

I am hyped for this. So hyped. I only finished watching the show again a couple months ago, too!

Shellsharks' recent post on burnout made me write this. I went into it writing it to post it online, then I decided I didn't want to post it online, but kept writing it for myself. Now it's written and I've read it back, it's going online.

So there we have it. Busy, tumultuous and anxiety-laden times. But also, good times. Hopeful times. I am not my job or my salary. I am my family. My family keeps me sane when I need it, and keeps me happy always. Firefly gives me excitement and hype. And the web we inhabit, the web we humans build, and the communities we foster give me connection. Those of us that aren't LLMs are humans with human problems, and the world certainly has a lot of problems right now that I haven't even touched on. Remember the human out there, as well as yourself. I don't think you can fight or fix or prevent burnout by doing more stuff, I think doing less of the things you aren't enjoying can help a whole lot, and allow you to find some stability again. I look forward to doing less in the future, whenever that may be!

Stay shiny, folks.

And Mike? Thanks for posting. Thanks for your email, for Scrolls and everything else you do. I see you. Fistbump.

Scraps #13

2026-01-09

Scraps #13

A bunch of scrappy notes from 2025-12-19 to 2026-01-09, posted on Friday 9th of January 2026 at 10:55

A bakers dozen!

Spotlight

Several in the spotlight this time around as I've had a few weeks of good stuff finding my eyeballs

Check this video out on what propoganda is, really.
Cory Doctorow on the post-american (de-enshittified) internet that we may have in front of us if we can just make a move on it
I know this is typically a technology/indieweb/infosec/etc oriented scrappy list, and yes, I'm not an educator, however this page about facilitating learning in the classroom is just too good to not share
Accessibility is critical - Innovation that excludes people isn't innovation; it's just shiny exclusion.

Things from the tubez

I'm not supposed to link to Luke's website, because it's too small.
James considers the indieweb in 2030
maren is as smart-phone free as they can be in the USA without being locked out of everything
'Living off the land' in infosec terms means using existing resources (built-in commands, drivers, applications, etc) to aid your attack. lolol.farm is a catalogue of the different living off the land types
Aaron is advocating for less js when html/css will do the same thing
Here's a beautifully woven telling of a writing club for both non-LLM writers and recovering LLM prompters by one of my favourite bloggers
Chuck is... self reliant! Chuck dives into the reasons for owning your own domain name
Dan has built an 88x31 button generator and it's pretty slick!

Scraps #12

2025-12-19

Scraps #12

A bunch of scrappy notes from 2025-12-01 to 2025-12-19, posted on Friday 19th of December 2025 at 22:34

Oh, December. The Seasonal Affective Disorder kicks into it's fullest of swings, things get colder and darker and greyer... But hey, I get to keep the fireplace on. Ahh, cozy!

what are the haps?

Hey! I haven't kept on top of the indieweb+infosec goings on as much as I like to this month, hence the delay in posting Scraps #12. As we reach the end of the calendar year in my part of the world, (most) people at work tend to... slow down a little. Which is good for me, because it means fewer distractions in the workplace, so I end up being more productive. The pattern held true again this year! However due to that productivity, I did uncover several... let's call them "issues"... which required much attention.

First off, identifying and fixing several (minor!) data breaches at work is always fun (despite being frustrating that they exist in the first place) and an opportunity to both learn something new as well as prevent future damage by applying lessons elsewhere.

Secondly, I identified a disgustingly simple but very severe breach in a third party platform... a platform whose sole purpose is to securely record data breaches! Eek. Thankfully the company was very receptive to my report, and rolled a fix out after about an hour. Pretty good for a fairly niche product!

Thirdly, and most importantly for me, the 'merger' of the organisation I work with into another org is still planned for next year. I've referenced it here and there but haven't written anything up about it as yet, so here's a stream of consciousness about it! There's a good chance I'll lose my job through this merger, however the work to get from merging to merged will be pretty intense. I'm not one to be sour about this - in fact, I think the merger is a good thing. I am looking forward to this work, because it'll be super interesting. And hard, I'm sure, but as someone who takes pride in their work I aim to make this a very smooth technological transition regardless of if I am dropped at the end of it. I want to point at this event as say "I made that slick." Maybe even use the new Org management for a reference or two if I do need to find another job.

The financial anxiety surrounding this uncertainty sure is kicking my butt though. I am preparing for the worst. We run pretty frugal at home anyway, but I'm already starting to put feelers out for roles elsewhere (not a great job market right now though!) and I am also considering other ways to generate a little side income to take the edge off a little. Something I've never really done before. I am considering having a ko-fi and/or LiberaPay link on this site on a /support page or something, but I haven't done that as yet purely because I really don't want to be holding my hat out to the crowd, as it were. But I might. We'll see.

The best way for me to deal with this anxiety is, according to my brain, stare at my budget spreadsheet for ages. This doesn't often work though, so instead I've been reading some cool stuff on the internet when I've had a bit of time to catch up! Here's some of that cool stuff:

Lyra has a new take on clickjacking with a very interesting svg technique
Robert writes about discovering the indieweb in a calmer way
If you want to learn both the intricacies of HTML/CSS as well as how forgiving it can be, there's no better method than reading a bunch of website source. This is exactly what Alex has done. Back in May, she wrote about what she learned building an archive of over 2000 pages
Icons are in menus everywhere now - Jim has posted about this trend and I am in agreement! It's something I had (perhaps subconciously) noticed. This post brought it straight into my awareness and now I can't help but sigh every time I see yet another menu stacked with icons
And speaking of things you don't know you know (or don't conciously acknowledge you know...) did you know that Google Maps algorithmically shows you restaraunts? I mena, yes. Probably, you did. This is obvious if you think about it - of course it does, same as everything else Google fires into your eyeballs! But honestly... I hadn't considered it when using Maps. What impact does that have? What does, say... the london food map look like de-Googled? Lauren did the heavy lifting and figured it out. The result? Way better results, if you're looking for more fair results looking for a restaraunt in London anyway!
The legendary Soatok has built out a reference implementation for the public key directory server: This software implements the Key Transparency specification I've been working on since last year, and is an important stepping stone towards secure end-to-end encryption for the Fediverse
Being a solo self-hosted-platform-maintainer-plus-developer isn't easy, as Kayla wrote about following an outage. Love the openness of this writeup, and it's interesting to see some of the problems solo folk face given the resource limitations (time, energy, money) plus the restrictions on their environment (solar powered server in a closet) (note: despite what the website says, this is a thing used by one person, it's not a company with paying customers)
Jyn is just havin' fun and findin' out
Our personal sites can be abused in interesting ways to generate profits for malicious folk. Ibrahim experienced this recently with their search page!
Taggart has taken the idea of webrings and wants to evolve it to the next step by adding reasonable trust. The concept is called Ringspace, and it's a mighty interesting idea!
A new (old) web browser to test your websites on - A load of Sega Genesis DLC (yep) have been recovered, including a web browser (yep again!) called the Sega Channel Genesis Web Blaster
Jake got hit with a crypto miner thanks to the recent next.js exploit, except they weren't running next.js! ...or were they...? Be aware of the dependencies used by the stuff you run, folks!

Bonus

I have a long list of stuff to consume - videogames, books, tv shows, movies, blog posts, web serials... the list goes on. One list I fail to maintain anywhere sensible and instead keep in my brainspace (that eroded, pitted, leaking mass of nuerons that somehow keep me conscious) is a list of Tools I Wanna Try Out. Some I remember, most I forget. Maybe I should start putting them here?

I love kanban, it changed the way I manage my work and I will wax lyrical about it to anyone who can't get away fast enough - Kanban MD is a neat little application that builds a kanban board from a simple markdown file. Looks super slick!

Popos 24 Spinning View In Games Fix

2025-12-13

Popos 24 Spinning View In Games Fix

Posted on Saturday 13th of December 2025 at 22:24

Just a quick bugfix as it took me a while to find a resolution to this issue.

If you're trying to play a game on Pop!_OS 24.04 (which I'm really liking, by the way!) and are finding the mouse is going all kinds of wacky in games launched by Steam, you might find your problems go away if you put the 60-steam-input.rules file (which can be found on the official Valve git repository) into the following directory: /etc/udev/rules.d/, then rebooting!

I've been working through my video game backlog for years now, and I'm currently playing through Dishonored. I'd just reached the final level when I foolishly decided it'd be a good idea to upgrade to the newest Pop!_OS release. The upgrade went fine, but launching the game after went... a bit wild. First the keyboard didn't work and the mouse was funky, but disabling controller support in the games properties on Steam fixed the keyboard. The mouse thing... yeah that took a while. Whenever I moved the mouse even a tiny amount, my view would spin down and to the left very rapidly. I'd spin around in a circle, what... a dozen times? More? I couldn't look up, I couldn't look right. Even on the slowest mouse speed barely made a difference!

To fix this I was watching dmesg, fiddling with mouse smoothing, trying various Launch Options in Steam, mucking about with the resolution and sensitivity of my mouse with Piper (which is cool! I didn't know this existed before this problem... silver lining and all that, eh?) and poking at the fact that my mouse - a Logitech G502 - was also reporting itself as a keyboard because that had seemingly caused problems a few years ago (it turns out needs to do for the macro support)... When I stumbled across a little reddit thread from 9 months ago with this fix. Full props to them!

Hopefully posting this helps some out there to avoid digging and searching the 'net for four hours to unravel the fix...

Scraps #11

2025-12-01

Scraps #11

A bunch of scrappy notes from 2025-11-24 to 2025-12-01, posted on Monday 1st of December 2025 at 22:04

People always come up to me and ask me: "Who are you and what are you doing here?" and to them I say: "Aaaaaaaaaaahhhhhhhhhhhhhhhhhhhh!!!!!!!!!!!!!!"

In other news, I've had an idea for a long-running REALLY SIMPLE web ..."game", and you know what it's probably been done dozens of times already. But I'm sort of trying to not care about that? I find that the knowledge that someone else has already done that (better than I ever could) is putting me off writing and building things. Alternatives are good! Why can't I listen to my own advice. So anyway I'm gonna build out a really simple fun time (and energy, eek) waster. Because why not?

Anyway! On to actual things that exist on the internet:

Spotlight

Julius has been posting one kick-ass homepage from the depths of the 'net - it's absolutely worth digging through the 50+ posts and keeping an eye out for future ones as there's some wild diversity (in 'website' terms, primarily!) in there and I don't know about you but I am here for it

Stuff? Things? Sure why not

This is a great blog post about maintaining an open source project - I'm a big fan of kanban and currently use the self-hosted open source version of LeanTime, however... Kaneo looks interesting. I will have to check it out!
Nick has a new site, and it has a /why page which explains, well, why nick has a new site. This is a great idea, exploring the why of things. I should add a /why page, just like Nick, Miriam, Mike and many others
A good take on the cloudflare incident a few weeks back, encouraging this kind of detail in the future will only make the experiences of one team propagate to other teams more easily, a net benefit to all
Regularly visiting web wanderers may know that I love reading post mortems. Generally these are of a technical "oh snap that thing broke in a special way" type documents BUT I also like reading gaming related stuff too! I saw Nishchal post to Mastodon about their design post mortem for their Catnap Chaos game. It's a good read, covering what goes into designing the mechanics of a relatively simple game (in a good way!)
Email, the ultimate weapon in this modern age. Declan writes eloquently about those angry emails you sometimes find yourself sending. Loved this.
Oh, whoops. It looks like it's time to shut our websites down. Personal websites are dead! ...obviously they're not, as described by both rina and fLaMEd.
Recently, Joel has written about finding it hard to do, whatever it is in that moment, and Elena feels similarly! I totally get this. Sometimes starting is the hardest thing. I get that way with this site, even putting these scraps together can be a trial. Not the doing. The starting. Which reminds me about that game-thing I mentioned at the top of this page... hmm

Bonus

You probably have those places on the 'net that you visit regularly. A space that, when you load in, you know what you're getting, even if the content changes there's a tone or a feel to the site that keeps you returning. For many people, it's the big social media places.

I have a few of these 'check several times a week' spaces. One of them is a web comic you've probably heard of, XKCD. I wanted to make special mention of a recent comic here that's powerful because of the overall webcomics consistency, the diversion from its typical topics, and the years-long wait for a followup to a few prior editions.

XKCD: Fifteen Years

Who's cutting ogres? Tell 'em to quit.

New Phone

2025-11-25

New Phone

Posted on Tuesday 25th of November 2025 at 22:14

In September 2021, I purchased a Pixel 4a to replace my Pixel XL.

Running with stock android for that time until now, and being very careful with the phone, I managed to keep the 4a running smooth and fast throughout it's lifetime. I had needed to replace the phone for a while, as it had been out of support since August 2023. This was a long time to run it without security patches, something that left me feeling more and more nervous the longer I left it, however life and family were flowing and money... wasn't. I didn't install random crap, I kept an eye on what few app updates I had available to ensure I could trust them to the best of my ability. Then came the dreaded battery update, but luckily I wasn't affected too much. My battery ended up lasting for a total of about 3 hours of light usage, and I'm keen to stay off my phone more nowadays, so the short lifespan didn't bother me too much. I should have replaced it, however my frugal mind got the better of me. Instead of replacing it in 2023, or indeed early 2025 when the battery update of doom came out... I held on.

Through the battery update renumerations, I was given a £100 voucher to spend on the Play Store on a new Pixel Phone. I was tempted to abandon Google entirely (for... well, all the reasons) but I've holding out for a phone that's... better. The new Nothing phone (3a) didn't appeal after I looked into it, and neither did some of the other new kids on the block (like the fairphone) - I wanted GrapheneOS, which requires a Pixel (for now!). But £100 off wasn't enough. I held out. Until the black friday deals. And I'm glad I did - A pixel 9a is currently £150 off, PLUS add to that my £100 voucher, that's a £499 phone with 7 years of support for £249. A pretty good price!

So, it was time. I did it. I ordered it earlier this month.

It arrived on the 14th of November.

I've (mostly) migrated my stuff over (stock android for now, GrapheneOS... soon, hopefully) and I'm... annoyed at the direction of Android but it's pretty okay, now I've mucked about with the settings somewhat and replaced the horrible launcher with something much better imo.

And then... this evening. This evening I crouch down, the phone slides out of my pocket, falling what is probably... 10cm? But onto a hard wood floor.

Crack.

Sigh. I'm so annoyed.

I can probably swap the screen out myself, but I'll be dropping another £100 on the replacement screen alone. I'm going to have to just live with it for now, maybe grab a few cheap screen protectors to see me through until sometime next year, what with the holiday period, car MOT (literally tomorrow) and some other events happening over the next few months that require monetary output. Money money money.

Oh, plus potential redundancy from $dayJob happening next year, but that's another story for another time :D

The moral of the story is twofold: Look after your phone, and Pixel 9a screens are fragile as fuheck.

Scraps #10

2025-11-24

Scraps #10

A bunch of scrappy notes from 2025-11-15 to 2025-11-24, posted on Monday 24th of November 2025 at 14:40

Ten whole scraps?! That sure is... a number! I expected to fully give up by now tbh but let's keep this cheese-wheel rolling down the hill(?)

When life gives you lemons, ask life "Oh hey, life is a person now. Where have you been? Why lemons? Why not money? Can I get some money? No? Oh, well... Can I get more lemons? Is this a one time thing or are you like on a schedule or something? Can you even talk?" and then make lemon meringue because we've got lemonade at home in the fridge already.

Spotlight

I love reading deep dives, post mortems and incident reports, so here's a treat if you too are that way inclined!

elite.bbcelite.com is a site dedicated to Elite, the 1980s space trading sim game, containing not only its source code, but also over 130 deep dives on the How's and Why's of the game!

Stuff and things!

A mix of ups and downs this week for the internet. Emphasis on the downs. The dead internet is still alive! For now, at least...

Rodrigo writes that AI/LLMs in Firefox appear to be something that nobody wants, but Anil disagrees - The important thing is using LLMs safely, and having safe LLMs to use. Can Mozilla fill that need with its integration into Firefox? Who knows...
Bruce has another javascript-free method to frustrate immoral web scrapers - the more the merrier!
Uggla is also getting up in the immoral web scraper fight with their own JS-free method and I love to see it
This GitHub repo will let owners of (some) apple air pods fully enjoy their earphones on non-apple devices. I don't own these but it warms me to see people unlocking the capabilities of a device that are unfairly locked down!
Cloudflare outages are always interesting, I really enjoy reading their writeups. Here's the most recent (at the time of writing lolol) writeup of the 2025-11-18 outage
On that cloudfront outage above, you know, the one that took down a huge percentage of websites and apps? Yeah, well rina points out that you're just introducing a single point of failure and you probably don't need it at all
No cloudfront outage will ever take fyr.io down (i hope?) because I self host! Laura is also a proponent of self hosting, I'm happy to see more people writing about it!
Careful when building things though, whether through bugs or intent you may find yourself abandoning resources that can not only be claimed by third parties, but are in fact still relevant and potentially trying to be utilised out in the world. This happened to Microsoft recently!
There's value in fixing bugs, whether you're removing old unused code, resources (see above) or just making the UX a bit friendlier or accessible. Consider a 'FixIt' - Bug-fixing for a week per quarter helps refine the product/service, feels good for the devs, and makes the users happier!

Bonus

Half Life 2 is my favourite game, and I love learning new things about it, such as how a bug meant an NPC toe broke the game, even time travelling to break older versions that weren't originally broken

Scraps #9

2025-11-15

Scraps #9

A bunch of scrappy notes from 2025-11-07 to 2025-11-15, posted on Saturday 15th of November 2025 at 22:02

I had an email from a web wanderer the other day who mentioned that they enjoyed these Scraps. That was super nice, I appreciated it more than they will know. If you read through my scraps, I thank you. You are awesome.

If you have a thing that you like to read, listen to or watch that a human put together, send them a little message. They'll thank you for it, and you'll make the human web that little bit nicer!

Spotlight

Proposal for a .meow tld? All profits funding the lgbt community? Great idea! Hope this works out: dotmeow.org

Bonus

Gaming - I enjoy it, but I'm limited due to my weak hardware. That may change soon though if this is as cheap as I hope...

Valve announces 3 (I guess they can use that number now) new bits of hardware - a PC, controller and VR headset. I run off an old donated laptop so this will be great for me if it's cheap. Also, so many hints about a new half life game in the announcement trailer and I haven't even been able to play Alyx yet *sob* regardless I'm excited for a potential reveal. 16/18th November? I hope so!

Scraps #8

2025-11-07

Scraps #8

No scraps since 2025-08-01 but hopefully I can start regaining some momentum, posted on Friday 7th of November 2025 at 22:25

I have a dedicated /scraps RSS feed, a dedicated posts feed, and an everything feed. If any of those tickle your pickle, peruse this very link!

A long period of busyness has kept me away from pushing out some /scraps - I haven't been keeping on top of the goings on as well as I'd like to. Life just gets in the way sometimes! It's not like I've done nothing - I've had some opportunity to do this and that. Besides, a scrappy schedule seems fitting.

A short one this time around - a good start to the week gave me some opportunity to play catch-up, but that positive start quickly devolved into chaos. However, at least I've got something to put forward into the 'net!

Spotlight

Sometimes you come across something that resonates. That's what we've got this time around, something a bit hard hitting, a pill to swallow if you will. But it's also enlightening and freeing! Perhaps it might be good for you?

You are insignificant. And that's not only okay. It's a good thing. This post speaks to something I've been trying to come to terms with for a while.

Links

We're keeping it contained in one small block today - some recent blog posts, articles and sites I've seen that I wanted to share.

Joan Westenberg (who crafted the post in the spotlight above) wrote up why they chose Discourse over discord, something I'm considering myself (see "Ideas for Life" section)
Related to that, Taggart-tech compiled and rated some discord-alternative-slash-community-building tools
Get yourself a tor mirror of your site, like the flower.codes site has done. A good idea!
Your software should identify when it's being run on corporate machines according to LGUG2Z. Obtaining funding from corporations whilst remaining open to individuals is an interesting concept that could be good for developers who would otherwise release stuff for free.
By adding complexity to streamline stuff in the homelab, Jana has built a prison for themselves. There's too much going on to quickly whip something together in a moment. This feels very indieweb (indiesysadmin?) - getting closer to the roots. There's a pleasure in doing things by hand. There's also pain of course, but that's part of the charm. Honestly, it is! Please believe me!
Related to the above, you should ask yourself: What the hell have you built? Keep it simple, kids!
Maintain web state via the URL where appropriate and useful - more sites should adopt this. I'm considering ways to nicely present this as an option on this site, for example for the post filter or theme, plus the RSS feed
It looks like XSLT will probably be removed from chromium
A tape (hopefully) containing a full copy of Unix v4 has been found - that's pretty cool!
The year of the linux desktop is finally nigh, for 3% of gamers are now using Linux! Huzzah!

That's all for now, folks! Catch ya'll on the flip side.

New Theme Selector

2025-11-03

New Theme Selector

Posted on Monday 3rd of November 2025 at 21:00

Last updated on Wednesday 5th of November 2025 at 21:09

I've added a theme selector to the top right of (almost) every page, and I think it's pretty nifty.

I set out to not use any javascript on this iteration of the site, primarily because I think that a website should be fully functional without javascript, but also partly because I just suck at javascript. As a result, I've long relied on the /themes page for web wanderers to set their theme on this little buried nook of the web. You click on the theme you want, and after a little reload (and cookie), you've got it.

That worked fine, except I don't think many people noticed there were themes there unless they went exploring or reading older posts.

A while ago I used the CSS :has selector to allow wanderers to filter the posts on the front page, and more recently to toggle summaries of the posts. I knew I could take advantage of this to set a theme, but it was a bit of a task as I had slapped together the themes quickly, each relying on elements of others that may get properties inherited and overwritten later. It was (and still is) kinda yuck.

I decided to test if I could combine :has with :checked and nest declarations. One hour later, I had a working prototype with two of my themes ported across. Some rewriting of the CSS as well as the theme application/detection logic on the server side was needed, but it wasn't as bad as I had feared.

A few days later, I finished up, tidied up some loose ends, added a little polish, did some testing (hold the phone) and merged the changes into the live site. Success! Thanks to a couple members of the 32bit.cafe discord for helping me test!

You can now use the theme selector - just select a theme from the drop down in the top right and the page should update to that theme right away. This will not be retained across pages, unless you hit the 'Save' button. This saves a cookie on your device, unless you select the default theme, which will delete the cookie.

Implementation

Using the :has CSS selector, combined with :checked (which works on dropdowns, whodathunkit?) and nesting the theme-specific declarations and selectors inside it, like so:

html:has(select#themeSelector option[value="redalert"]:checked) { /* CSS here */ }

If the 'redalert' option is selected in the drop down, it'll apply whatever CSS declarations are in the curly braces. Let's break it down a little:

html:has(

I've chosen the HTML tag here instead of the BODY so I can set a rule on the HTML (like a background colour) as well as the BODY. I do this for the Paper themes. So, if the <html> tag has...

select#themeSelector

...a <select> with the ID themeSelector (<select id="themeSelector">)...

option[value="redalert"]:checked

...with an option with the value of redalert currently "checked", which is poor grammar but in the case of a <select> dropdown list, means "chosen" or "selected" (but we can't use :selected because we already have ::selection and that's way too similar) we can...

/* CSS here */

...Apply our 'redAlert' theme CSS! We just write out our theme-specific CSS declarations like our colour variables and any other selectors and declaration blocks. This changes depending on the theme of course, with some being small and others being quite a bit larger in size and complexity.

We can embed nested CSS selectors and declarations inside this, too, allowing us to change the property of elements for particular themes. If you can hack looking at some... probably not great CSS, check out this sites .css file here.

Time to make more themes, I guess!

Problems

Let me know if you have issues.

Clicking the Save button doesn't do anything! - yes it does, it redirects you to /themes?choice=[themeChoice] which sets a cookie, then sends you back to the page you came from. Unfortunately, or fortunately depending on how you look at it, this happens so fast that on a decent latency connection the browser doesn't even flicker. It looks like it's not doing anything at all. I don't like the idea of setting a false delay, so I need a way to signal that the theme has changed. Can I use :active or :focus to do this... a task for the future

HTML, Emoji, and Large Language Models

2025-10-15

HTML, Emoji, and Large Language Models

Posted on Wednesday 15th of October 2025 at 22:44

I describe a way to prevent Google Gemini (and probably others, I haven't tested many) from consuming your website content on this post. I am not claiming that this works when Google or whoever train their models initially, I don't have a way to test this myself. It does stop Gemini from being able to read the content of a page when you ask it to, though.

I kinda came to this dynamically, as outlined below, and I strongly suspect that I am not the first person to consider or test this. Honestly, I haven't looked. Intentionally. I wanted to explore this blind and see what fell out. Personally, I enjoy reading about the journey someone takes to come to a finding or conclusion, whether it's worldview-changing or (as in this case) something mundane. However, it's not for everyone, so if you want to skip to the working method without reading about the journey, head to demo4 onwards.

The Journey

I saw two things online ~~last week~~ back in July (oh man this has been in my drafts for so long) that got me thinking.

The first thing I found myself reading was the WHATWG HTML spec. I can't remember why I was reading this, but I happened across a particular section which deals with custom elements. In short, did you know that you can use emoji in HTML elements, as long as you follow some other rules? No, me neither! This means that <banana-🍌> is a valid HTML element. Neat! Add to that the overbearing friendlyness of web browsers and you're pretty much left with the fact that, despite the earlier requirements, almost anything can get away with being an element.

The second thing I read was what turned out to be a poorly edited LLM-generated slopfest of an article. I don't have the link anymore, but the title had "Goodbye SEO hello GEO" in it. There's loads of articles with this, including a book (or a few, I don't know and don't care) proclaiming that SEO (Search Engine Optimisation) is dead, and that GEO (Generative Engine Optimisation) is the new in-thing. I mean, whatever. SEO was and continues to be irrelevant for pretty much the entire web thanks to the enshittification of the bigger search engines, and me personally because I just don't care. And GEO? Who wants that? In fact, where's the AGEO (Anti-Generative Engine Optimisation) at!?

At that moment, I wondered if I could use custom elements to become invisible to LLM scrapers. My theory - and I have no idea how they obtain and prep data to train on - is that they're probably written to be efficient and make a bunch of assumptions - sucking down the entire internet is something that will be resource intensive, so if they can reduce impact by essentially greping their way through <p> and similar elements, maybe that can be taken advantage of?

Now, step one with any theory is determine if someone else has done this before, but in the interest of something to do (and write about) I opted to skip that step and go straight to building a demo and testing it. In fact, I have done just that. If you use a screen reader or other method of consuming web page content... I apologise for what I can only guess will be a complete disaster if you check out any of the demos linked below. I'd be interested in hearing about your experience actually... Here we go!

🔗Demo 1 - HT(e)M(oji)L

Let's start at the start - in HTML, many elements are defined already. But as we learned earlier you can pretty much use whatever you want. What if we use (where appropriate) custom elements that are one character and an emoji?

Demo 1

Take a look at the source to verify the emoji are present and correct. I've even slapped together a rudimentary CSS file to highlight that you can style custom elements in exactly the same way you style pre-defined ones. Just be aware that the browser won't have any defaults for them, so you'll need to really work on setting them up to behave as you want them to. Hey, this would have been much easier than using a reset.css back in the day wouldn't it?

What happens when you push this into an LLM? Let's find out, but first, I want to quickly summarise the LLMs that failed for some reason:

Duckduckgo-hosted models (gpt-5 mini, gpt-4o mini, gpt-oss 120b, llama 4 scout and claude haiku 3.5): they won't connect to a URL directly
Proton's Lumo: same as above
ChatGPT 4: As above, surprisingly. Maybe grabbing URL contents sits behind an account or paywall, barf. I ain't about to find out

And as for the long list of models that worked out:

Google's Gemini 2.5 Flash & Pro

Eesh, that's a short list. Ah well. I'll stick with Gemini for now as I have access to it through work.

Feel free to push these demos through an LLM yourself, let me know what they say if you do. I've only tried those listed above and I'm not going to sign up, pay for, or hunt around for others to use. The prompt I used to get the LLM to check the URL is:

Actual query: Please take a look at this URL and describe the contents. Do not search the web via a search engine, do not consider the words in the URL to determine the purpose of the page, look only at the content of the link I provide. The link is: [URL goes here]

Gemini 2.5 Flash

No luck there - it's picking up the visible content. Perhaps the Language part of Large Language Model is winning out, parsing the raw data and plucking out the stuff that makes sense to it. Like, you can ask an LLM a question with a typo and it'll still know what you're asking. Maybe it is ignoring the odd characters like > and random words and letters in amongst actual sentences, treating them like noise?

Gemini 2.5 Pro

The Pro LLM worked here, too. Interestingly, the first time I ran Pro on this demo it flailed about a bit and ended up telling me it was about a document from the Dole Archives at the University of Kansas. The document appears to be a political text from a campaign. The text criticizes a congressman named John Bryant for voting to increase taxes and for voting himself a pay raise. It then proposes a different political vision that includes cutting the budget of federal agencies (except for law enforcement), eliminating agencies and programs, reducing taxes, and reforming the welfare system to make holding a job more attractive than being on welfare. The overall theme is a call for a change in the direction of the country towards "prosperity and progress." - slop. It wasn't until I went back and recreated the prompt to get a screenshot that it actually worked. Must have been having an off-day.

🔗Demo 2 - Element noise

Elements can be named anything, but they can also have any number of attributes (for example, <span title="hi!" twist chosen="the one true entity"> has title, twist and chosen attributes) - do LLMs parse language, or markup?

We can find out by reading documentation, code, searching the internet or making some basic assumptions, but instead, I'm going to throw another bit of HTML at it and see how it responds.

Demo 2

On this demo I have written a bunch of fairly random and nonsense sentences in each of the elements, but the page itself renders... okay, more made up nonsense, but at least it's understandable and follows a theme. The hope is that the LLM will just see a mishmash of random sentences and not really know what to make of it:

Gemini 2.5 Flash

It's pretty clearly understoof the page. Well, darn, I guess they're parsing the markup, which makes sense. There would be loads of noise (inline styles, javascript, comments, etc) otherwise! Interestingly, Gemini 2.5 Pro had a little more trouble figuring out what was going wrong:

Gemini 2.5 Pro

It failed on the first try, but when prompted to try again it did correctly understand the contents. When viewing the 'Show Thikning' contents, we see that perhaps the "Browse tool" can't quite extract sensible content, but the second attempt requests the raw text content, where seemingly it can properly "render" away the elements. Interesting!

🔗Demo 3 - Body vs Brain

If I were to write a tool to grab web page content, I'd very likely extract the title of the page, perhaps some meta data from the <meta ...> tags, drop everything else in the <head>, then push the contents of the <body> on and parse it to pull out the content. So, let's try taking the content we care about out of the body.

As mentioned before, web browsers make some huge allowances to the HTML they're given, rendering things that perhaps they shouldn't, or clobbering together broken markup into something it can show the viewer. This is a bad thing in my mind, but does make web weaving that much more accessible, which is a good thing! We can abuse this little "feature" and literally just chuck the important stuff we want to show users into the <head>, put crap we don't care about into the <body>, hude it with CSS, and let the LLMs suck down the useless content whilst the humans get the good stuff!

In theory, anyway...

Demo 3

So, the page looks like this - two sections, the first which you can read plainly is a quick letter saying that a two-day work week would be swell. One can dream. This content is actually in the <head> of the HTML. The second section, a short story I slapped together, is in the <body> as the gods intended, but you can't see it in the browser (unless you view-source) because I've hidden it all with CSS.

Gemini 2.5 Flash

Once again Flash has pulled the content out, but has parsed both sections. Looks like my way of ignoring the irrelevant-to-the-content stuff LLMs might care about is not the way the smart people do it. Oh well.

Gemini 2.5 Pro

Pro has again failed on the first try, but when switching to a simpler "raw data" mode concluded that there are indeed two sections. We wanted it to ignore the first. Alas.

If these LLMs are (generally) having success parsing HTML, what if we moved the content out of the HTML? LLMs won't care what font is used or what colour the background is, right? What if we literally pulled all of the content out of the HTML - no walls of text to distract, no font-size: 0em, nothing but raw, empty markup?

🔗Demo 4 - ::before abuse

Demo 4

In this edition, I've created a CSS file, which has a 'translation' dictionary of classes, each corresponding to a letter of the alphabet and a few symbols. The HTML file is just a load of <span>'s with a class that represents the letter that should be in its place. Check out the source, including the CSS file. It's easier to see it than it is to describe it.

When the page is rendered in the browser, the ::before is responsible for inserting the actual letter I want a human to see before each span. I hide the single letter that is in the HTML so as to not infect the view of the human, however this is what I want the LLMs to see. The result is a human-readable page, but one that is essentially empty of content (Z's aside) and hopefully unreadable by our LLMs.

This does have some quirks though... the page renders a little... off. Maybe it's font choice, maybe it's something to do with some kind of dynamic letter sizing going on, something to make text more readable? I don't know. I'm sure it can be fixed, but another downside is you can't highlight any of the text. Probably not that big of a deal for most web content, but hey. Let's see how our plucky little LLMs deal:

I also decided for this demo to exercise my long-dormant short story writing muscles and... I got a little carried away with it, continuing it with demo 5 later.

Gemini 2.5 Flash

Success! It's not rendering CSS, just grabbing the page contents and trying to parse the HTML.

Gemini 2.5 Pro

What's with the try-hard pro model needing to be told to try again? Either way, the second time around it did get a result, but it was the junk one!

But... imagine trying to write this? Don't worry, the internet is an amazing place. I was discussing these demos on the 32bit.cafe's Discord back in July and a member followed up three days later with...:

🔗Demo 4 - special shout-out to Lera!

Lera has built an awesome tool to automatically generate pages with the demo4 ::before method - check this stuff out: lerariemann.nekoweb.org/var/obfuscator.html!

Not only will it automatically translate HTML into this LLM-proof noise, it'll ALSO optionally add markov babble in, randomise the HTML tags AND add a de-obfuscation script as part of the output!

THIS IS AWESOME! Thanks Lera! :D

🔗Demo 5 - simplified

I was playing with this idea more and wanted to simplify it - why ::before-ify every letter when we could instead just process each paragraph? It'll be way easier for LLMs to parse, but I put together demo5 to show that it also works this way, too.

Plus, it's an excuse to write chapter 2 of the story I started in demo4.

Demo 5

This is a much simpler version of demo4, a made up element represents each paragraph but is descriptive enough for a human to be able to parse in the HTML/CSS a bit easier. It's feasible for a human to write this, moreso than demo4. Not that you need to, with legends like Lera hanging around!

Gemini 2.5 Flash

Gemini 2.5 Pro

Neither Flash or Pro Gemini can fully render and parse the page, as with demo4, which is good. The advantages of this is the relative simplicity from demo4 of course, but also the output rendered in the browser is more uniform than demo4's off-kilter-ness. Still can't highlight it though. Oh well.

🔗Conclusion

So, it's possible to hide content from at least Gemini, yet present that content to humans without messy javascript, proxies, caching tricks or other awkwardness. Just a little bit of HTML and CSS. Well, awkward HTML and CSS, but hey at least Lera has you covered on the generating of the code. But only for now - Demo5 will become readable pretty quickly if needed, and I imagine that if it became a problem LLMs would just start literally screen reading rendered browser output, which I know happens with "agentic" AI or whatever it's termed now, but will likely become an option for other general LLMs soon enough.

Host Change

2025-09-23

Host Change

Posted on Tuesday 23rd of September 2025 at 21:32

To have your own website, you need a domain and somewhere to put the files that appear when you type that domain into your web browser. It can get a little more involved than that, but there's loads of help available. There are thousands of options for your host - some free, many paid - but I have been with Gandi for years now. I always liked their No bullshit stance (which was also their slogan for a long time) - their tools worked, their services were good, pricing was reasonable. To be honest, two of the three are still true (in my experience), however the last, the pricing? It has gone up too much.

It all started back in 2019 when the Montefiore Investment purchased Gandi. Nothing actually happened, at least not from my perspective. Things stayed the same. I was surprised - from the moment the purchase was publicised I began considering my move away, but I waited until things took a turn. And they didn't! Confidence was shaken, however I only really had a few little projects and domains that weren't of much value to me personally. This site existed as a Wordpress blog, but I wasn't really too attached to it.

Then in February 2023, Total Webhosting Solutions acquired Gandi. They started to charge for their mailboxes, which was entirely acceptable - managing mailservers is a PITA at the best of times. I've done it, it wasn't the easiest thing to do in 2009, and I can't imagine it got easier in the decade-plus since.

But then they hiked the price of their domains, and their hosting, and it became a point of contention. I paid, but I didn't like it.

Then I needed to transfer a domain. One I owned on behalf of someone I know AFK, a transfer from one Gandi account to another. It went fine, but there was an issue with the mailbox payments. Despite not being on my account, my payment details were still being used - they had transferred with the mailbox instead of inheriting the payment details on the other persons account, and I couldn't take them off myself.

I contacted support and eventually got a short, rushed response which essentially blamed me for this (which, okay, maybe I was supposed to remove payment details from the account prior to moving a domain?) and then just told me to talk to the new owner, and closed the support ticket. It wasn't exactly a bad experience, but it was kinda... bullshit?

No bullshit, yeah, right.

So anyway, I finally did what I've known I've needed to do for over two years now and migrated to something else. Something better.

No longer am I on a wierd mid-point between a VPS and a shared web host (which, honestly, worked fine for me) - I have a "Proper VPS" with Hetzner, and it costs like €4 a month. I've still got most of my domains on Gandi, but I have tried porkbun, which was nice, though I'm going to look for an EU registrar at some point.

The migration of files worked a treat - previously I would need to SFTP the fyr.io site files over when things changed, which I didn't mind. It's kinda oldschool, and as you can probably tell from my content I'm a fan of the older web, warts 'n' all. Now however I am taking advantage of Gitlab pipelines to push changes here whenever the main repository is updated. It's way more hands off, which I do appreciate.

And, thankfully, the site worked right away - everything pulled over fine and picked up right from the repo, no data loss as far as I can tell.

There were a couple minor things though...

The site didn't have SSL for about an hour, as I had to wait for the DNS changes to propagate before certbot could validate the domain. Easy fix, just had an exercise in patience. I'm sure the three people that look at this site occasionally didn't notice anything.
More annoyingly however, the following day I got dozens of notifications on my RSS feed reader app indicating that all my posts on here were unread. I still haven't figured out why that happened, as far as I can see everything is the same! If you are subscribed to my RSS feed and got hit with this, I apologise.
I also had an issue with the /void content - it didn't come across, a victim of the .gitignore preventing changes to the required files. Because I'm not too shabby at this whole computer thing^{(citation needed)}, I had up to date backups, so it was easy to pull the new file and change things around to let it work with gitlab runners rather than my manual SFTP process. An easy fix.
Last up, I had an email from someone who had scanned the domain with a vuln scanner during the migration, picking up on some poor SSH algorithms that were in use. I hadn't had a chance to check that out since spinning up the VPS, so that was great! It's not too big of a deal, but it is now fixed and they have been credited on the /vulns Hall of Fame - Thanks for emailing, Gaurang!

So, what's next?

Well I'm not too sure. I want to resume my scraps when I have more time in the week to stay on top of the indieweb and fediverse goings-on. I've also found myself playing more video games recently, my way of switching off. I'm getting in maybe half a dozen hours a week which is pretty decent for me. Maybe I should write mini reviews or something, though I only really play old games. My laptop can't run much that came out after 2020, and I've not spent any money on games in years.

Other than that, I'll start tinkering with the site and some of those projects you have on the backburner for years. You know the ones. We've all got them. Oh, actually, one thing I do need to do is migrate some other fyr and non-fyr stuff across to Hetzner... I should do that before I need to renew my other Gandi VPS.

If anyone has recommendations on an EU-based registrar, or notices a problem with this site, hit me up!

The Months Of Contention

2025-09-23

The Months Of Contention

Posted on Tuesday 23rd of September 2025 at 16:49

It's been a busy August & September, but I've finally managed to find some time (and energy) to work a little on the site.

As you may have noticed, I haven't been keeping up with /scraps, my link roundup posts. I got to 7 editions before a hiccup in the schedule, but that hiccup turned into a choke and now it's suffocating somewhere far down on the todo-list. I've just been too busy to keep up to date with the goings on.

Work has been pretty hectic, as it always is this time of year. The August break is our opportunity to knuckle down and smash out projects we've been building up to during the rest of the year. This year had extra challenges, as we needed to migrate our server infrastructure to a new location with zero budget due to some pretty major power work happening at the site. We had committed to keeping everything digital working, and happily this work was planned and executed ready for the two weeks of no electricity, which ended up only being a one week long event.

This went well from our perspective - everything got moved where it needed to go, minimal disruption. Hosted services stayed online (mostly) and we had no issues or incidents... except, well, we ran off a generator for that week, and half way through the week, at the end of the day... it exploded.

Yep. And, yes, there was only one generator - the cost for redundancy was too high, given the temporary nature of the power situation (and the fact that almost everyone except for us had this time off or worked from home that week) so everything crapped itself. Luckily, UPS kept things stable enough and we had no breakage due to power surges or anything like that - the servers went offline and by the next morning they were back up humming along nicely.

Following the completion of the power works we hit the ground running, getting as much done as we could, and, sustained by nothing more than grit and momentum, we reached the end of August, at which point everyone else turned up to start their jobs. Most of those people had just had five weeks off, and came back fresh faced to see our bleary bloodshot exhausted eyes staring back at them.

As is tradition, in flooded the tickets - a few hundred within the first few weeks. Not out of the ordinary. But it was busy. Not hard, just... frantic, I guess. Things have calmed a little, I typically write off September as I spend most of my time jumping into the support queue and taking on the more annoying issues. People don't believe me unless they've experienced it for themselves, but there are holiday gremlins - they follow you around and watch you fix up all the machines and projectors, configure all the new switchess, tidy up all the cables, test all the new patch panels, build out your ACLs, then on the morning of the first day, those gremlins come in and just fuck your shit up. Suddenly, thirty of the three hundred things you know were working the day before are just fucked for no reason at all. So you rush around, fix them, then the next day a new 10% of your previously working shit is ballsed up. I swear, these gremlins haunt me. I've never seen one, but when I do, I will catch it, and I will ram a punch-down tool so far up it's... anyway, things are a bit calmer now, I'm ready to start easing my way back into web stuff. I've already done a few bits, another blog post coming soon.

Until then!

Scraps #7

2025-08-01

Scraps #7

A bunch of scrappy notes from 2025-07-25 to 2025-08-01, posted on Friday 1st of August 2025 at 15:00

Pinch, punch, first of the month.

This week I've had a bit of time off work and am still not up to date on my reading list. I'm starting to think that maybe there's too much cool stuff on the indieweb?

Unfortunately, thanks to some utterly stupid laws that are finding their way in to society, for this edition you are going to have to verify your age. Please submit a 3d face scan to verify that you're between 5-155 years of age. If you fall outside of this age range, would you kindly close your eyes until you've closed this website. If however you're within this age range, or a sentient being of any species other than human, welcome. Enjoy.

Spotlight

It's the 1st of August today, which means it's Blaugust!
It's also HTML Day TOMORROW (Tuesday 2nd August)

Indieweb, Fediverse & Social Media - people stuff

The enshittification of governance continues - laws are being set to serve the government instead of the people. It has been coming for a long time.

Now the internet is reacting to the age verification laws that are being passed with dismay, resignation, and new VPN subscriptions. It's the resignation that will allow these things to pass - things like it have been tried again and again and this time it's not been shot down. That's the secret. Keep trying and eventually you will succeed, whether you're an entrepreneur trying to start a business, or a government trying to implement a privacy-invading slippery-slope law.

Soatok outlines the issues with the age verification and explains that, no, it doesn't need to be a privacy concern... unlike the current state of things in some countries
If you're fed up with online marketplaces (ebay, amazon, etsy, etc) you could try selling the epic stuff from your attic on your own website perhaps?
Sometimes you have some old tech laying around but don't want to sell it or chuck it in the recycling just yet. Annie has repurposed an old ipad for consuming blog and small web stuff
If you're wanting to participate in the small web, you'll probably need to learn HTML! HTMLHobbyist has been shared around this week. If you're interested, check it out!
Manuel has now interviewed 100 bloggers as part of the People and Blogs series, and writes about this experience
LibreOffice developer Mike has had his Microsoft account banned - I'm sure there's two sides to this tale but it sure looks bad on the MS side
David has drawn three panels that reflect the web as it was, the web as it is, and the web as it will (hopefully) be. Let's work to get there!

Infosec, sysadmin & code - tech stuff

This week I had to remotely upgrade a 12r2 hypervisor running a domain controller straight to server 2025 (do not pass go, do not collect $200), running on a single generator (no redundancy). There is a failed disk in the array to boot. Talk about risk! It worked out though. Now I just need the replacement drive to arrive to the site...

Hacking used to be done just because - it was for fun, for curiosity. Make something do something it wasn't intended to do. It is of course still like this - case in point, how about hacking a washing machine?
In the spirit of making something do something it isn't supposed to do, here's a twist on that. Taking software designed to be as fast as feasible, and making it as slow as possible? Why? Why not! Here's Jacob keeping themselves occupied by making Postgres reeeeaaaal slow - love it
Dating app Tea was breached recently. They stored info such as scans of ID and selfies, which has been plucked from the internet and shared around. So many fails here, *facepalm*
Pwn2Own has a $1,000,000 reward for WhatsApp exploits, and have detailed more info about the categories and rewards on the ZDI post
Google's Project Zero team are changing how they announce discoveries - they're adding in a 10 day annoucement that something has been found to their 90+30 days publication delay. Interesting idea!
An attempted bank heist was going to play out like an episode of Mr Robot with their use of a Raspberry Pi and interesting evasion technique

Bonus

The Virtual Moose - gaming related links and blog posts
CopyParty is a new-to-me file server which looks pretty cool actually. I haven't tried it yet as it was just shared on the 32bit.cafe discord earlier today, but going from this new youtube video alone it looks... pretty darn good.

Scraps #6

2025-07-25

Scraps #6

A bunch of scrappy notes from 2025-07-18 to 2025-07-25, posted on Friday 25th of July 2025 at 15:00

What are the haps?

We're rushing through July and things aren't slowing down. The weather here is... variable, shall we say. Gone are the days of consistency, we're getting high winds, blazing sun, heavy rain, calm overcast skies and lightning in a single day.

It's unreliable! But hey, at least I've got the consistently high quality of the human web to keep me feeling positive, and the consistently poor state of information security to keep me feeling negative. They balance out, right?

Spotlight

I have had the legendary Pierre Carrier's website bookmarked for many moons, and keep coming across his awesome stuff. He just posted that he has, via xmit.dev, launched Let It Be Known, which is a service that offers free subdomains for anyone that wants 'em, and it's super easy to manage to boot. There's another thing from him down in the Tech section but I wanted to spotlight libk.org and encourage you to check out his website and writing because he is top tier
Luke emailed me about his new /blank page, a delightfully blank page (except for the wonderful addition of some flowers) which I have added to my Intentionally Left Blank post!

Indieweb, Fediverse & Social Media - people stuff

My list of posts to read and websites to browse grows faster than I can consume them. At this rate I'm going to need to employ a content curator!

Will has a chop - a personal seal - that is used to stamp art he creates. But he had a thought... Why can't this apply to digital stuff, too?
Moderating a community hard work, as highlighted by Mark. Support your mastodon instance team if you can!
The privacy of Firefox has been a given for a long time, perhaps you shouldn't trust it though? Evan doesn't think so
David thinks people should surf the web again! You know, instead of living inside the corporate web. And he has three quick guidelines to make it a more enjoyable and meaningful experience
@hyde has a great series of interviews called Over/Under, where he seeks out the opinions of a specific web denizen regarding particular things. This time around he's asked Elizabeth Tai about digital gardens, social media, rss, books and tea
Alan Levine gets nostalgic for one of the earliest html tutorials on the web, something he himself put together back in 1994!
And on internet nostalgia, who still uses text based (ascii) smileys o.O ? I do :D and so does Adële!

Infosec, sysadmin & code - tech stuff

It's been a busy week on the tech front, with some pretty significant infosec-related events.

MDN (Mozilla development network) celebrates 20 years with cake, a tradition from browser vendors during the nostalgic early Firefox days
Pierre (that brainbox from today's spotlight) has been digging into mouse latency recently. He then dug into it some more, which included building a tool to get even better data, and has gotten deep into USB land.
Some SharePoint admins struggled to sleep over the weekend as ToolShell was used to compromise on prem instances across the internet
An outsourced, third party tech support org allowed credential resets without verifying identity. Yikes.
I was playing around with some HTML & CSS stuff, trying to prevent LLMs from scraping website content for an upcoming post the other day. I discussed some of the experiments on the 32bit.cafe discord server and Lera took an idea that emerged and built an online obfuscator tool which automates this for you!
When it comes to pissing off LLM scrapers, @ache has written up how to build and serve a HTML zip bomb
Google launches OSS Rebuild, a project that builds open source packages and compares the resulting resource with the published one, enabling automated confidence that a package is what it says it is
Using OSINT, researchers have put together the structure of a Russian SIGINT unit, using their commemorative badges
Of course we can't go a week without hardcoded creds in a widely used product, can we? This week it's HPE Access Points
I love peeking behind the curtain of a big organisations technical infrastructure. Netflix has just posted part 1 of how they got live TV to work
Yorick is dealing with the frustratingly loose HTTP spec

That's Scraps #6 concluded for this week. Keep posting awesome stuff, peeps!

Early Blackberries

2025-07-21

Early Blackberries

Posted on Monday 21st of July 2025 at 15:16

Since moving into a rural location I've found myself picking blackberries most years. This typically happens in late August and September. Not this year, though. I noticed a load of ripe blackberries in some weeds in our garden (I'll get to it eventually, I swear) and found 10 minutes to pick some yesterday.

This is very early, right? The climate is clearly changing, it doesn't take early blackberries for me to notice this, but it is yet another indicator of change.

Admittedly there aren't too many here, though this was one small area of the garden that I've left unmaintained this year, I barely had to move to pick this lot. I have a somewhat hidden location down a public footpath that I like to visit to pick blackberries at, I wonder how many I would get from there?

Scraps #5

2025-07-18

Scraps #5

A bunch of scrappy notes from 2025-07-11 to 2025-07-18, posted on Friday 18th of July 2025 at 15:20

Scrap number five, alive! Happy Friyay to all, but especially for those of us with a work or school week that ends on a Friday.

It's been a busy week. I honestly thought this would be the week I didn't get Scraps out, but here it is, and it's a big one!

Thank you to my subscribers, patrons, ko-fi donators, the mafia, and to the venture capital funders that I've sold out to for allowing me to automate all this with the AGI of the week. The previous sentence is entirely false.

Oh, whilst I'm here, are you aware of any indieweb/smallweb/humanweb podcasts? If so, please tell me about them! I would like to know what's out there already ^{<span class="hide-me">before I seriously consider publishing my own scrappy podcast</span>}!

Let's get on with an entirely human (if I can be called that?) curated list of just SOME of the cool stuff I've read over the last week:

Spotlight

That's right, TWO in the spotlight! Shush, I make the rules here.

Blaugust is coming!
Michael emailed me to share their linkshare posts and I enjoyed them so much I'm sharing it on with anyone who doesn't yet have the site bookmarked or added to their feed reader! Thank you for your email :)

Indieweb, Fediverse & Social Media - people stuff

The indieweb has been alight with stuff and the quality never disappoints. To be honest, I could have filled this section with so much more but I have loads of posts and articles still on my reading list to get through! People have been busy typing stuff and I'm here for it.

Check out this CSS blur image technique (thx to @slightlyoff for posting about this!)
What's that? You want more awesome CSS stuff? How about this game of Tic Tac Toe, built in CSS? Yes, that's right, there's no HTML involved at all! (Only works on desktop browsers!)
John has redesigned their site and written a post detailing it and some of the decisions behind it - If I may, it's lookin' good!
Evan has written a post about how they write software quickly - I'm not a programmer *at all* but it's great to read stuff like this, how the pro's do it. Inevitably there are lessons to be learned from people experienced in or masters of their craft
Mozilla is trying something new, asking the community directly what it wants from the browser in the future so if you are so inclined, this may be your opportunity to encourage Mozilla to not destroy firefox and encourage them to work on useful stuff instead of the, let's say... less useful stuff we've seen over the years. This asking the community thing... is a new idea? Hmm. (This should perhaps go in the tech section below but I'm putting it here because it's asking humans about what humans want from the browser)
I love that there are publications about this web we enjoy and weave - Here's another magazine about it!
The accelerated use of LLMs in every nook of technology is changing society akin to how the power loom in the hands of those with power caused instability
Tom writes about LLM inevitabilism which is helping to pave over individualism and human creativity on the way to the future that capitalists want
Saint has written a post about a peeve they have regarding the indieweb - I'm guilty of this but also have the same peeve! I must do better.

Infosec, sysadmin & code - tech stuff

Special thanks this week go to energy drinks for keeping my brain switched on during another infosec-heavy week.

haetae on the 32bit.cafe discord server shared graphite, a free online graphics editor which is already pretty cool yet still only in alpha! Oh and it's open source!
mcdonalds kept 64 million job applications locked securely behind the password '123456'
Back in 2023 Derin found a bypass to Chrome's MV3 "oh you can't use an adblocker anymore because $$$" feature which would have allowed for adblockers to continue working (for a few days, probably) - whilst not worthy of a reward nor is it highly technical, it just goes to show that yes, there is often a way around a limitation. Especially when old legacy code is involved
Keeping track of software End of Life dates can be painful - Easy, Cheap, Good - pick two. However one neat site that I reference pretty frequently is endoflife.date
In yet another example of incompetence from the current admin of the USA, a DOGE (read: *dodgy*) employee "published a private key that allowed anyone to interact directly with more than four dozen large language models (LLMs) developed by [..] xAI"
AI bug bounty slop is overwhelming open source projects, including curl. What's the solution here? I've seen suggestions including requiring video evidence, a token payment that is refunded upon valid submission, and more, but each has tradeoffs. Are those worth it?
Crawling 1 billion pages in just over 24 hours is possible to do cheaply in 2025, but it has some interesting considerations to take into account

Bonus

I am pondering whether to have a dedicated section for games and other categories, or keep a bonus section for stuff that doesn't fit into the Tech or Human categories (which already overlap a bunch!)

This week we're feeling inspired, I strongly encourage you to check out each of these links, if you are so inclined!

The ever inspirational and genius Ben Akrin reflects on 10 years being off grid
Dan Hollick is building a reference manual that describes how computers work, in an incredibly detailed and beautiful way. The first part is out, How does a screen work? It's a thing of beauty!
I love reading incident reports of technical outages and exploits. I've just learned that wikimedia publishes their own incident reports at wikitech.wikimedia.org
When it comes to FIRE (Financial Independance, Retire Early), I'm all about the spreadsheets (and clearly the daydreaming about it is addictive as it's super unlikely I'll ever get there :'). It's probably a little unhealthy, how closely I monitor our finances at home. Kyle did the same, but wanted better. From humble beginnings to success - Kyle had an idea, a financial independence planning app that was better than the rest. So he built it. He's just hit $1m yearly recurring revenue and has written a bit about it. I enjoy reading success stories like this, being able to create something and get an income from it is a dream of mine which feels out of reach. He has this advice: Whether you’re building a business, just getting started with investing, or working toward financial independence, it’s often the small, consistent actions that compound over time. Just like dollar-cost averaging into index funds, showing up consistently to improve your craft can produce surprisingly powerful results on your path toward a better future. ^{(Note: this is not a sponsorship or advert! I hadn't actually heard of Projection Lab until I saw this blog post linked by someone. I just liked the success story!)}
Although a different industry, this is similar in nature to that finance app blog post above. Here's a long youtube video that goes through how Expedition 33 came to be, and it's a bit of a tale to be sure! We've got a small team creating a game bigger in scope that teams of thousands can produce, we've got coincidence, luck, skill and passion, culminating it what will probably be game of the year! Though I don't know that, because I haven't played it yet... One day I'll save enough to buy a modern gaming machine :D

That's it for Scrap #5! The fifth edition? Volume? Zine? Whatever, it's Scraps. It is supposed to be scrappy and inconsistent. The only consistent is the quality of content that you lot keep putting out into the world! So thanks for that!

Do you have a website or blog? share it with me!

Scraps #4

2025-07-11

Scraps #4

A bunch of scrappy notes from 2025-07-04 to 2025-07-11, posted on Friday 11th of July 2025 at 21:40

We're four Scraps in! Momentum!

To celebrate this audacious occasion, I've created a dedicated /scraps slashpage AND a dedicated scraps RSS feed, which you can find listed on the new /feeds page!

I've also changed the scrap title be an incrementing number instead of the date it's posted. Yay or nay? Or perhaps doing something like 25-28 ([year]-[week number])? Hrm...

Whilst I ponder this, let's get right into it.

Spotlight

Call for content! @xandra is looking for content for the next issue of good internet magazine - do you want to contribute?

Indieweb, Fediverse & Social Media - people stuff

Hey, have you seen those, uh, what are they called? You know, down on that small h2o ocean planet with the plastic pollution and melting ice caps? Come on you know, the ones going through an extinction level event with massively reduced biological diversity, who move metal with explosions and think it's a good idea...? Yes! Humans, that's it! Have you seen what a small subset of one very particular niche of them has been up to this week? No? Check it out:

Louie thinks the web is so cool. I agree, and I'm not alone, Kai thinks so too! The secret is to find the people, not the products or robots :)
On that note, The indieweb has got ReadBeanIceCream down. Down for more, that is!
It seems that for ReadBeanIceCream, "more" involves detailing what RSS feeds are all about. Check out this great guide on RSS
RSS feeds have a long history, relative to most internet-based tech. It's not the only thing with history though - A few folks have been looking back through time at the evolution of their websites:

John travels back in time and looks through the websites he's had since starting back in the 90s(!)
10 years is a long time too, Murray has looked back over the decade and explored the evolution of theAdhocracy.co.uk
Ruben has been posting for 1 year now. That's not as long as the others above, but it is a grander achievement in isolation in my opinion. The first post to your site is often the hardest, but keeping that momentum for a year? That takes dedication, passion, care and effort. Ruben has celebrated this milestone with a thank you post, aimed at the human web, filled with links to fantastic sites you should check out! I am honoured to be included in the post. Ruben, thank you for making the human web what it is today! I look forward to the next year and beyond!

If you're all kinds of inspired by those people and what they've created, perhaps you want to make your own site? well you're in luck, because Joe has been reminded of a fantastic resource for learning html!
Ah, HTML, what a delight. Did you know that August 2nd is HTML day? And did you know that there are events that happen around the world to celebrate it and spread the hypertext-based joy? HTML events?! Well, now you know!
And here's another important day: The Electronic Frontier Foundation has celebrated its 35th birthday this week

Infosec, sysadmin & code - tech stuff

Beep you say? Well to you I say, boop.

WatchTowr throws another great post detailing the latest Citrix vuln. Déjà vu?
Staying with WatchTowr, they've published another post detailing a pre-auth SQLi which leads to RCE on the FortiWeb Fabric Connector. Y'know, the thing that ties Fortinet security things together and consolidates them. That thing. Sigh.
Adrian has taken an LLMs hallucinations about a feature that SoundSlice's music sheet scanner doesn't have and created that feature to fulfill a demand in the market - pretty neat idea
A bunch of browser extensions have been identified which started out seemingly legit, but once popular got infected by malware via an update

Bonus

Whatever you do, do not tease the sheep.

Ben is living a dream, and just the other day had another magical moment where he saw a deer and its fawn
Check out these sweet Pokémon card effects made with CSS! They're not recent, but I've only just seen them. They look so great, and just with CSS?! Awesome.
If you enjoy reading Scraps, have you considered ...reading Scraps? Yes, I am of course not original, as Alex has been posting Sunday Scraps for nearly 100 editions! I love finding scrappy little linkdumps. If you have one you post yourself, share it with me!

Scraps from 2025-06-27 to 2025-07-04

2025-07-04

Scraps from 2025-06-27 to 2025-07-04

Posted on Friday 4th of July 2025 at 22:15

The third scrap! Whoda ever thunk it'd reach the lofty heights of three editions? Not me, that's for sure.

This week, I'm happy to tell you that Scraps is fully sponsored by absolutely nobody and contains no product placement or adverts! Ain't that great?!

Once is once, twice is a coincidence, but thrice? Well now, that's a pattern. I guess I'm going to have to carve some time out this week to sort out a dedicated Scraps RSS feed, as well as /scraps slashpage. I mean, the header already links there and it 404's so I guess it needs to happen? I'll add it to my todo list!

As for your own todo list, perhaps peruse the presently provided posts post-haste? I don't know, you do you. BUT we do have some cool stuff to read this week!

Spotlight

small_cypress has started Small Web July - cutting out walled garden social media, the 'big web', and scrolling forever in the month of July

Indieweb, Fediverse & Social Media - people stuff

Because people are people too, y'know!?

With large language models still churning out a lot of slop, their integration into everything continues. What does this mean for the future of the web?
when the tech giants cut their connection to the web - with AI summaries and walled gardens - they freed us. There's little point farming backlinks or bowing to their demands now. And in their wake of destruction I see space to grow. So lets establish a small, human, sustainable web of independent sites. For ourselves and any friends that will join us. - Homestead the Web by @caolan
@elazar misses the internet
The old internet is still here - we've wrapped up June, and that means we get to take a look at the take two posts that have been written for June's Indieweb Carnival!
It's now July! New month means new Indieweb Carnival theme, and this time around it's totems hosted by Maxwell
The lack of self awareness of the wider web never ceases to amaze, Aram spotted this beautiful and disappointing piece of irony on The Verge
@Nina raised a good point... what happened to my Status option on IM applications? Sure there are some current implementations, but there are too few!

Infosec, sysadmin & code - tech stuff

By welcoming our robot overlords, you're actually just welcoming the billionaire who owns them. And that, to me, seems like a pretty shitty deal! Tech is still cool though - like all tools, they can be used or abused.

Speaking of cool tech, Tech Greatness The Internet Archive is expecting to archive their 1 trillionth page in October. To celebrate, they're holding a party and a live stream! Be there, or be square! ...or watch an archived copy afterwards I guess that's cool too.
689 Brother printers were in the news recently for having easily guessable passwords - Terence Eden speculates that they may therefore be illegal in the UK. Interesting thought!
Let's encrypt has now stopped email notifications - less infrastructure for them to manage, which is a good thing!
Staying with Let's Encrypt, they have now issued a cert for an IP address and will begin rolling this out to prod sometime this year
There's a new method to extract different content from the same .zip file, depending on the tool used to extract the files
Every public commit to GitHub is archived in GitHub Archive, including ones developers try to delete (because they contain secrets, for example) and you can easily extract them with this writeup and tool

Bonus

It's a good idea to bring ourselves out of technology and websites occasionally.

That hasn't happened this week, as I've been busy reading about game development! For no reason I tell you, no reason at all.

Two similar Dev logs from two different games were posted online recently, going over efficiency gains related to multi threading. Whilst I don't have the nuerons to fully understand each detail, they're nonetheless fascinating and I certainly appreciate a good, deep technical analysis of a problem. Here's the Arma 3 OpRep, and the Dyson Sphere Program post

Scraps from 2025-06-18 to 2025-06-27

2025-06-27

Scraps from 2025-06-18 to 2025-06-27

Posted on Friday 27th of June 2025 at 23:20

Last week was a poor week to start these scrappy little notes, because I've spent this week on vacation! Most of my infosec- and tech-related reading happens at work these days, so this edition is a little short on that front.

You'll be very glad to hear, if you haven't already, that Mike has returned to the throne with not one, not two, but three scrolls! What a relief, welcome back Mike! I can, after a single issue of Scraps, finally stop posting these :)

Wait, hold up. What's this? In the latest Scroll, Mike made a special mention of these scraps?! And LIKES it?! Well, dang.

Mike, thank you for the shout out and kind words. You've inspired a lot of what I do here on this site, and I share your opinion about human curated lists and the intentional decline/enshittification of search. The human touch is increasing in value and rarity on our ever-more LLM-driven web, so continue to publish I shall!

So, dear reader, if you're here for a scrappy collection of tangentially related and poorly organised links to stuff I found cool or interesting or worth sharing, read on. If you want something way better, check out Shellsharks' Scrolls (after ;)

Let's get the second Scrap going!

Spotlight

Xandra is at it again - she has reached version 1 on Riverton, a free browser based virtual pet and ranch simulation game - go check it out!

Indieweb, Fediverse & Social Media - people stuff

The human side of the web - some say the best side and I agree with them darn it.

If you're a fan of the fediverse, Jeff Sikes has your merchandise options covered with this awesome list
Jelloeater has consuming loadsa blog content down to an art, they write about the process here. Some management around this process would be useful for me since starting these scraps, and there's plenty of ideas for me in this post
One of the blogs I check every day, rachelbythebay, has been quiet of late. However, Rachel just posted a neat little tool, a rollover calculator which lets you determine the source of some of those magic numbers which seem to appear everywhere in tech for some niche reason
There are three things a man must do before he dies: plant a tree, father a child, and write a book. -- ReadBeanIceCream posts about hearing this quote and the realisation it unlocked within. It's a great read. Now, where's that book draft I wrote a decade ago...?
That reminds me, June's indieweb carnival prompt is "take two" by Nick. I hope you post! (And I hope I do, too!)
Speaking of indieweb events, this month is Junited - Ruben has shown his appreciation by listing some blog posts that interest him
There should be more events that encourage humans to do stuff, fight off the AI/LLM slop that is invading the web. But remember, you don't need an event to create something, whether it's written or drawn. And who cares if it's bad?! Bad art is GOOD now.
Publishing something is hard though, whether it's a book, a post, or artwork. Many of us find it difficult to post things online. SenFlyer writes about the fear of being seen.
Humans, we persist despite it all, and are still the best way to connect people, like this Mastodon thread by Andy Bell which connects freelancers, contractors and clients. Just like in the before-times! Ah, sweet nostalgia...
Nostalgia for the before-times is a powerful thing. I just learned about this project to make a youtube front end which looks like it came staright from 2009! Thanks to Jake for sharing this!
Time, and UI design, marches on, though. You may recall about 84 years ago (wait, it was only three weeks ago!?) Apple unveiled their new Liquid Glass UI. Well, it didn't take long before someone made it in CSS - only working in Chrome right now, mind.
Speaking of Liquid ~~Vista~~ Glass, Louie Mantia isn't sold on it, and I don't think many others are either... we'll see how it develops, I guess.

Infosec, sysadmin & code - tech stuff

When sysadmin is your job, you tend to switch off when you're on vacation. Which is where I've been for over a week! So we're a little short in this section this time around.

Cool tool: Nuclei - A free, open source, easily customisable, high performance vulnerability scanner. I've only just heard of this but I had a little play with it today and it looks pretty sweet.

Bonus

Some extra stuff. Why? I'll tell you why! Because, that's why!

Nexus Mods has been sold to a Venture Capital firm and let's be honest, that rarely goes well. Mod creator Leilukin doesn't hold out much hope for Nexus Mods to get better thanks to this.
If you question why you recycle when a hundred private jets attend bezos wedding, Thomas has the answer
.PNG is back - not that it went anywhere really! A spec update has arrived and... it looks pretty good actually.
And finally, here are a few more blogrolls for your consumption:

Light on the tech and infosec side this week. What, you wanted consistency in these scrappy little roundups? Oh.. you did... okay, I'll do better. Next time :) Until then, if you have any suggestions, feedback, wishes, dreams, homemade cola recipies, or bad jokes, please do let me know.

Scraps from mid May-ish to 2025-06-18

2025-06-18

Scraps from mid May-ish to 2025-06-18

Posted on Wednesday 18th of June 2025 at 23:00

As many of you will know, Mike Sass (shellsharks) runs an amazing "weekly newsletter / link roundup / information digest at the intersection of the IndieWeb and the Fediverse, with a splash of Cybersecurity stuff" called Scrolls.

Unfortunately for whatever reason, this hasn't been published for a month now. Hope you're doing alright, Mike!

In the spirit of "imitation is the sincerest form of flattery", I've decided over the last few days to have a little go at doing my own version. It's not anywhere near as fancy or comprehensive as Mike's esteemed Scrolls. It's more like a patchwork collection of links and notes. Little... Scraps...?

I don't know how often I'll post these, whether they'll change format, and I'll probably stop if and when Scrolls returns to the throne (I'm just keeping the seat warm after all.) With any luck I'll get something out somewhat frequently in the meantime. What's that? What's my schedule? Schedule Schmedule! (but fr it'll probably be on a Friday I guess? What do you think?)

If you have any suggestions, criticisms, cool links or anything else, feel free to say /hello.

Anyway, onwards and downwards!

Spotlight

mononoir has been posting loads of their awesome mspaint art, check it out. It's blowing my mind!

Indieweb, Fediverse & Social Media - people stuff

A collection about the humans that use the technology - cool things they've done, opinions, studies, etc.

Xandra of 32bit.cafe fame has launched the Good Internet Magazine and it looks amazing! I really need to order one of these!
The Internet Phone Book also launched! It is a collection of personal websites, each of which has been assigned a 'phone number' allowing you to discover, peruse and explore them by dialing on the site. The physical book looks so great!
Matrix is suffering from capitalism and Alexia believes its future might not look so bright. However, there are some open alternatives. They could each use donations though, including Matrix
Robert Kingett respects his beta readers, but not when they disrespect his work and feed his content into an LLM
Communicating securely is more important than ever, and this is especially true in some parts of the world. Cory has got your secure chat needs sorted, with, of course, the Signal app
If you're one of the many individuals who live in the space in the venn diagram where 3D printing enthusiasts and fediverse enthusiasts overlap, then rejoice! For 3dprint.social is now live - a 3d print hosting fediverse instance, built on manyfold (which you can use if you wish to self host instead!)
In a totally expected yet still sigh-worthy development, Social Media is now the main source of news for people, according to the Reuters Institute, with watching video content being preferred to reading text. Traditional news outlets are generally biased of course, and undoubtedly influence was exerted on communities decades ago ('propaganda') but The Algorithm ramps this up to the extreme, and the big personalities will have their own influences and agendas without any journalistic standards or oversight to keep a lid on extreme corruption and falsehoods. This can't go wrong... right?
But not all social media is bad - mastodon has been recognised as a 'Digital Public Good'
LMM tools produce code, but for Miguel it's not worth it. At least, not right now anyway. David goes on to share his opinion that LMMs are replacing human interns' work in a way that's a bigger resource drain than simply training a human intern
In related news, a small study shows that brain work less when LMM used four riting fings. Which makes sense, the brain is like a muscle and LLMs ease the burden in some cases where the human outsources learning to it
Benjamin has readers, even though that wasn't what he set out to do originally. If you write, someone will read it, but writing for yourself first is a great tactic for building momentum

Infosec, sysadmin & code - tech stuff

Technology itself is also interesting to me! Here's some neat stuff I've come across.

If you're looking for a way to steal data from a company, why not just send an email and ask for it? With any luck, the email will be parsed by Copilot, which will follow your instructions and ye shall receive!
Google Cloud IAM went down and took a whole load of other stuff with it - Yes, the internet is still a precarious stack of duct tape, glue and hopes & prayers

Google's Incident Report on the above

Programmers write or encounter bugs all the time - Henrik logs them all and has reviewed the last 9 years worth to see what lessons can be learned
LibreOffice joins the "End of 10" conversation and heartily recommends any Windows users to consider switching to Linux + LibreOffice instead of potentially needing to dispose of a perfectly fine device just to use Windows 11. And I must say, I agree!
Europe-wide takedown hits longest-standing dark web drug market, Archetyp Market, which stood the test of time operating for over 5 years. Crime doesn't pay! For more than 5 years anyway.
Neat infosec tool: Subdomain takeover with second-order
If you're looking for a PoC for cve-2025-33073, here you go!
Long before the internet, some phone networks were hackable by playing a single tone at 2600Hz. Whistled (by a human!) into a phone, it could grant you unrestricted access. Do you have the vocal chops to be an old-school phone phreak?

Bonus stuff

APOD is 30 years old! Here's to 30 more years of awesome daily pictures!
First images of the south pole of the Sun, which is pretty cool if you ask me
The SNES Development Game Jam is now running, until the 9th September!

That's all for now. I've missed countless awesome things but this has been a bit of a test to see if it's something I could put together once let alone regularly. The next scrap (if any?) will contain things from this point in time onwards and should be easier to keep up to date!

As mentioned earlier, if you have any suggestions, feedback, wishes, dreams, homemade cola recipies, or bad jokes, please do let me know!

A Most Eggsellent Adventure

2025-05-27

A Most Eggsellent Adventure

Posted on Tuesday 27th of May 2025 at 22:23

Our chickens are growing up so fast *wipes away prideful tear* and are now 16 weeks old. One of them - but we do not know which - just laid their first egg yesterday!

This is pretty early for this breed, they normally start laying somewhere around 20 weeks, but 16 isn't unheard of. We're glad it's starting. We have learned that we should get an egg every 24-26 hours, which is more than I initially believed. Three chickens, once they've all matured enough, should get us somewhere around 2-3 per day for a decent chunk of the year (they slow down when it gets colder) which is pretty good going for us and should leave us with a small surplus to give away occasionally.

We did, of course, boil it and eat it. A first egg can sometimes be pretty small or not quite formed right - luckily this one was fine although a bit smaller than we expect to get out of them, as you would expect what with it being the first. And yes... It was delicious! Sliced in half, bit of salt and pepper. Glorious. Not at all rubbery, the yolk was a strong and deep yellow, and it was very smooth texture wise. We are looking forward to the next!

Aaaaa Sometimes You Just Wanna Scream Into The Void

2025-05-10

Aaaaa Sometimes You Just Wanna Scream Into The Void

Posted on Saturday 10th of May 2025 at 23:01

Sometimes you just want to scream into the void. Release frustration, anger, energy. Show the universe how you really feel. Or maybe you just like to scream. Whatever works for you!

Well, instead of burying your head into a pillow to scream, screaming from the rooftops, or screaming under water, why not digitally scream into the /void?

Bring your A-game. No, literally - only upper and lower case A is allowed. There are some other limits but I'll let you get e-pissed-off when you hit them :D

Atera Leaked Their Customers To Mailinator

2025-05-09

Atera Leaked Their Customers To Mailinator

Posted on Friday 9th of May 2025 at 22:38

Atera is a Remote Monitoring and Management (RMM) and Professional Services Automation (PSA) tool used by many organisations to maintain endpoints and respond to support requests. Typically used by MSPs to access and support their clients, it has also found life within internal tech support teams thanks to its per-technician pricing model.

There was a slow data leak of active Atera customers (as in, the identity and helpdesk mailbox of customers of Atera - the MSPs that use Atera for their clients, or the companies who use Atera for their internal support, not any data belonging to the customers or their clients,) available to the public for at least 6 months. I noticed it in October 2024 and reported it, and it has been fixed as of May 2025.

It was, on the surface, a low priority & low impact information disclosure issue that affects the users of Atera who set up and test their SMTP config. It leaks no more information than the mailbox used to send emails, but a side effect (read: malicious opportunity) of this is that you can of course determine that a company is using Atera without interacting with them or Atera at all. It hasn't really got much of an impact outside of some timely social engineering scenarios, which I'll run through towards the end of this article, but thought that this was interesting enough to write up, given the potential damage that could be caused, and that it's basically just a silly implementation of a mail check.

The details

As with most RMM/PSA tools like this (of which there are many alternatives) you can configure Atera to send emails not from their email, but instead from your own chosen email address. One way it achieves this is by allowing you to input mailbox SMTP credentials, which it will then use to send all email. It will also monitor this inbox, automatically making tickets on its built-in helpdesk portal based on these emails.

Within this configuration page is the option to test that it's working. I wondered how it tested this, so I hit the Test button, opened up the Sent items on the mailbox, and...

Unfortunately, Atera implemented this test using a public mailinator mailbox.

Mailinator is a service that provides you with a completely open mailbox. As in, you don't need a password to see the emails. It's useful to sign up to websites or services you don't really care about. All you need is an alias (the bit before the @) and you can open any @mailinator.com mailbox. By design. It's a great service with paid options that offer many additional features.

As you can see in the above screenshot, when testing SMTP via Atera, Atera sends an email from your configured mailbox to mailinator, likely checking the result of this in the background to determine that the mail has arrived. To be specific, they send it to atera_test_email@mailinator.com, which you can view here.

If you were to sit and watch this mailbox you'd see mail flow in occasionally as admins configure and test their SMTP config.

Mailinator offer a free private mailbox, which works exactly the same except the mailbox isn't visible to the public. If they had registered and switched over to this private mailbox it would probably be mostly fine.

This is a problem, although it is a pretty low priority and low impact one, but I thought it worth writing about as with many things like this, add a bit of imaginations and there is a risk here. Interestingly the emails in the mailinator inbox get removed fairly quickly, though this is inconsistent. I suspect Atera cleared out this mailbox in the background on a schedule.

Theoretical abuse

Let's daydream a basic, theoretical phishing attempt using this knowledge!

I imagine someone nefarious monitoring this mailinator mailbox and, upon reciept of an email, firing off their own email to the email address that sent the test email, which will most likely appear right into the ticket queue on Atera of the customer who just tested it. This email could pretty easily look like it's from Atera themselves. Perhaps it celebrates them configuring their SMTP settings correctly! Maybe they could offer them a celebratory free month subscription? Something enticing. All they need to do is click this link to verify their account and the discount will be automatically applied.

An unaware tech (or perhaps even the account owner who set up SMTP) may be tricked into clicking this - they literally just set up SMTP, how would a bad guy know that? Their browser loads the link to an atera-lookalike domain. They login and submit 2fa (because they trusted that email, remember?) and now the bad guys have easy access to what could potentially be an MSPs client base, which could reach into the hundreds of separate companies and organisations, tens of thousands of endpoints, or more.

Reporting Timeline

2024-10-22 - Initial report - automatic reply acknowledging reciept of email
2024-10-29 - Reported again, this time to an additional mailbox
2024-10-29 - Atera support desk ticket raised. I am offered (automated) to create an account to monitor the progress of the ticket, which I do.
2024-10-30 - Support desk ticket closed, no comment, no communication
During the next month or two I periodically check the mailinator mailbox and continue to see customer mail flow into it, but then the holiday period happened, life happened, and this issue fell out of my mind for a little while...
2025-03-05 - I remember this issue! I check the mailinator mailbox: still not fixed. So before publishing this, I submitted another followup. Just in case it wasn't dismissed, missed or ignored again.
From here on out, I see lots of background activity on the ticket. The title is updated to include the [security] and [concern] tags, later with [platform], and the Last Updated metadata field sees changes every few days and people and departments are assigned regularly. Clearly, things are happening.
2025-05-07 - An update to the ticket is made, informing me that the issue has been fixed by the security and development team 🥳

At this point, I ran some tests and confirmed that the issue has been resolved. They now send a test email from the mailbox back to itself and to the individual who requested the test, which is perfectly fine. Mailinator does not see any mail.

This concludes this little episode of a minor but abusable Atera data leak! I'm trying to get back into the world of infosec in a more active way (I've been passive for a long time) and have a couple other things I'm working on. Hopefully I can start to build some momentum!

Spring Chickens

2025-04-16

Spring Chickens

Posted on Wednesday 16th of April 2025 at 22:02

We acquired some chickens back in February and they've been doing really well.

They spent a good long time indoors, but we were keen to get them outside quickly. At first it was because we wanted them to be able to roam the garden, explore, rummage through plants and stuff. Quickly, however, another reason to get them out surfaced.

They stink.

Okay, they don't stink stink, but their droppings smell pretty bad occasionally and it catches you by surprise. You'll just get a waft of chicken feces up your nose suddenly and wonder why they're in the livingroom. And that's even when cleaning them out at least once a day.

Anyway, they're outside, they're happy, we have them set up in a very nice coop with a large run, and let them out of it whenever we're home.

It's spring, so of course we've started to get the garden ready for planting food and flowers. The three chickens have, however, taken it upon themselves to claim a ceramic plant pot as their own. They wriggle themselves into it and kick up the soil and have a dirt bath, sometimes falling asleep. At least, until one decides it's not quite comfortable enough and wriggles, setting off a chain reaction of flapping, kicking dirt and b'kahk! until they eventually settle in again. It's pretty amusing.

We think they'll start laying eggs sometime in June. Until then we're working on getting the garden set up for planting, which I'll post about soon.

CSS Naked Day 2025

2025-04-08

CSS Naked Day 2025

Posted on Tuesday 8th of April 2025 at 22:30

The 9th of April is CSS Naked Day. If you're reading this in a web browser with CSS enabled on the 9th April, you can probably tell that something looks different thanks to the banner across the top of every page and... well, the lack of CSS everywhere (almost - I intentionally use inline-css in a couple of posts, such as the 'Don't trust copy and paste' post which would break without it!) Thanks to timezones, this site switches off the CSS styles a bit before the 9th, and they stay off until a bit after.

I've spent a little bit of time (though nowhere near enough) over the last year making this sites' HTML more betterer but I have loads of accessibility changes to make - keyboard navigation, testing text-to-speech (or simply recording a reading of each post), and more - whilst ensuring I adhere to standards as rigidly as possible. I will also be making the HTML more accessible to people who wish to learn, inspired by jamesg.blog. I will be adding comments to the HTML and the CSS, as well as eventually publishing some information on the backend and my process of getting content live on a /colophon slashpage, but I have some work to do, getting this site to a place I'm happy with, before doing that. That work starts at the beginning though - allowing as many people to consume content with as little friction as possible. CSS Naked Day serves as a reminder to me to work towards that.

I encourage you to accommodate as many accessibility methods as possible whether you work on websites or apps or indeed physical objects. If you need a reason beyond it's the right thing to do, then... it can only increase your audience, your readership, your customers.

Pink Trees

2025-03-21

Pink Trees

Posted on Friday 21st of March 2025 at 22:39

We're lucky enough to have three cherry blossom trees of some description at the back of our small garden. Around this time of year they explode in pink, and it's very striking. It doesn't last long, very soon the leaves will come through a deep red colour, but whilst we have it I try to take a moment every day to enjoy it.

Pictures can't do it justice, and I've posted them on this site before (a while ago, on the previous incarnation. Hrm... I haven't finishied migrating all those old posts over yet...) but here's one anyway.

I really should clear those vines away...

CSS Naked Day 2025 Is Next Month

2025-03-07

CSS Naked Day 2025 Is Next Month

Posted on Friday 7th of March 2025 at 21:37

Last updated on Tuesday 8th of April 2025 at 22:14

It's CSS Naked Day on April 9th - a day to remove CSS from your website to promote web standards!

I was too late for it last year, but I'll never miss it again as this site will automatically disable CSS on and around that date and I encourage you to do the same.

Checking out your site when it's naked forces you as a website owner to really think about your HTML and to perhaps spend a little time making it as compliant as you can!

Speaking of which, I think I have pretty solid HTML on this site, though I do need to spend more time making sure the site can be consumed by all manner of tools. Not everyone uses a typical web browser to browse the web after all!

When I wrote the CSS Naked Day code last year I also made it function as a theme. If you wanna check out what this site looks like without CSS, just click it in the /themes page (it's at the bottom.)

World Wild Web

2025-02-25

World Wild Web

Posted on Tuesday 25th of February 2025 at 23:19

The web used to be a wild place. Every site you visited had something interesting, something unique about it. Perhaps a funky design, a strange structure or format, or nothing - no design, no structure. Just stuff! Websites had character, each presenting its content in a way that only that site could. Websites had... personality.

Now, however, for many denizens of the internet, that personality which we could use to illuminate the world is confined and diminished and restrained and molded. The commercial web is what most people (in particular young or recently connected people) see and have seen for nearly two decades now - That web is devoid of different - It is built to support a rigid structure, red tape, policies and guidelines and you can't do that and no, do it my way. You're locked in. You're stuck, you're watched, you don't move, you're unable to bend and flex and be yourself. You are simply a resource that is supposed to generate content for others, fitting within the rigidity of the system to in turn keep others and yourself further trapped and tracked.

You have one purpose on the commercial web - to feed and be fed by The Algorithm.

The web is not the commercial web. That commercial web is but the surface, the outer layer forced into place through money and profits and demand for control and influence. Seen first because it is on top; You witness it and it takes hold of you and keeps you in its addictive grip, training you to not dig or swim or roam, you are to merely tread water. You are not to dive and delve elsewhere and find the wild.

That wild? The untamed, chaotic, wonderful real web? It still exists. It's here, it's growing. Blooming, blossoming. The unique voices of tens of thousands of human beings, their opinions, their endeavours, their creations, laid out into the internet just because in their own style with their own voice.

It is likely I am preaching to the converted, but if just one person can see this and dig a little, look up and see what the web should be. There's more to the internet than corporate interests and profit and algorithms and AI slop. The internet is for you and not for them. Now, more than ever, unique voices and perspectives are needed across the entire planet. Diversity, respect. You.

You don't need to compete, you just need to be entirely you. Human. Free from the bindings of the commercial web.

So get a website or a blog. Write something. Draw something. Say something. Build something. Rewild the web.

The world wild web needs you.

Intentionally Left Blank

2025-02-20

Intentionally Left Blank

Posted on Thursday 20th of February 2025 at 22:37

Last updated on Thursday 19th of March 2026 at 22:17

I made an intentionally blank page - /blank.

I saw Blake Watson's /blank page after they posted it on mastodon earlier. I thought it was a great idea, and the reasoning behind it (as per this archive.org archived website (the original site is still online but the why page has been removed)) resonates: the primary reason to offer internet wanderers a place of quietness and simplicity on the overcrowded World Wide Web-a blank page for relaxing the restless mind.

More so than ever does the internet, the www in particular, need places of calm. Free from the chaos, the algorithm. The "indie web" is that, as is any community of humans discussing or building things they're passionate about. But sometimes, you just wanna... stop, y'know? Only exist for a bit. Breathe.

Ahh.

The original site above used to have a list of members, 36 sites which also intentionally made a blank page. I checked each one of the personal websites current state out and found the following had their blank page still online:

matthewweathers.com
purcus.ch
akaharry.com
leftblank.com (is this cheating :D)
eyeshot.net (you may need to view-source, it doesn't render in the browser for me)

The rest had sadly either been removed/redirected or the domain taken over. Google, Yahoo and some others used to have fun and show nothing when searching for this, but the internet isn't allowed to be fun anymore, according to that boring lot.

Make a /blank page. Intentionally. Bring to the internet your own little slice of nothing, and let me know if you do or if you have one already and I'll list it below.

Other blank pages from around the 'net

Some other bloggers or webweavers have let me know that they too have a page intentionally left blank on their site!

jamesg.blog/blank
blog.darylsun.page/blank
rhzartist.neocities.org/test/blank
benji.dog/blank
mihobu.lol/blank
shannonkay.com/blank
flamedfury.com/blank
lukealexdavis.co.uk/blank 🌷🌹🪻🌸🌺🌻🌼💐🥀🪷
r74n.com/blank
riri.my/blank {"opinion":"Awesome!"}
I've just (2026-03-19) found that the indieweb.org site has a page dedicated to blank pages! Neat!

SPF And DMARC At Primary Schools

2025-02-19

SPF And DMARC At Primary Schools

Posted on Wednesday 19th of February 2025 at 15:40

A few months ago someone I know working at a primary school was having email troubles, and happened to mention this to me in passing. I took a look and noticed no SPF or DMARC records were present. Whilst unrelated to the issue, this was concerning. SPF and DMARC are pretty basic things to get in place which help prevent spam being sent from your domain, and they don't cost anything either. I passed on my thoughts and left it at that.

But it kept nagging me... if this primary school was pretty bad on the SPF/DMARC front, are any primary schools doing a decent job? Are most? None? I envisioned a malicious actor emailing from a nice friendly local school domain, convincing locals (including parents of course) to just click this link or donate for this trip... Or worse.

I decided to find out.

Skip to the results if you want the good stuff

Scanning all primary schools

The first task is, of course, knowing what to scan. What are the primary schools in the uk? Can I even determine their email domain?

Yes, because the uk government maintain a public list of educational institutions! Not only that, but they make it available as a .csv file which contains loads of data, primarily the school type (primary, secondary, etc,) a link to their website (if any) and whether they're currently open. Handy!

Now, I'm only doing this rough, this isn't a scientific study nor am I seeking perfection and complete data accuracy, it was a bodge of a few powershell scripts knocked together so make allowances for the quality :)

I downloaded the biggest .csv file from the gov.uk site and generated a much nicer .csv file containing the 16708 currently operating primary schools. Of these, 308 didn't have a website, so they're removed from the list.

I have my target list ready!

Though wait a moment, you eager beaver you. From experience, I know that a very large percentage of school website domains do not match up to their email domains. This is due to several reasons (renames, council-issued domains, Trust membership, etc) - suffice it to say, I can't rely on the website domain being the same one emails come from from that particular school. So I did the only thing I could think of - I wrote up another little script to grab the html contents of the school homepage and parse it for an email address.

Of the 16400 schools with a website listed, 3632 of them had some kind of error when I tried to grab their website data (website timeout, parsing problem, 500 error code, etc) or I couldn't find an email address on the homepage. I should note that if I didn't find an email address on the homepage, I also parsed the HTML for a link with "contact" in the URL, grabbing that pages' HTML and parsing it for email addresses, to try and catch sites that only publish an email address on the contact page (of which there are many).

My approach wasn't perfect; A bunch of these websites look like they have valid email addresses available when viewed in the browser but they hide the actual address behind javascript to deter spam (or deter annoying people like me). I couldn't be bothered to unpick or defeat these, so they're excluded from the results.

I originally just regex'd email addresses out of the html, however I was catching a load of obviously-unrelated email addresses (like in HTML comments containing license or contact info for a javascript library) as well as other random not-email-addresses, so I resorted to the easier but less complete method of looking for a 'mailto:' link. This has further excluded many results, as many sites I scanned don't wrap their email address in an anchor tag:

<a href="mailto:email@address.tld">email@address.tld</a>

That left me with 10344 (probably) email domains, which is pretty good. I should add that I only took the first 'mailto:' link, so in here are some pages where they publish an email address that isn't owned or used by the primary school, but perhaps to a third party. Unfortunately, a large percentage of school website domains do not match their email domain, so these shall remain in the data as I won't be going through unpicking them.

I wrote my own parser to scan these records, which worked sort of okay but not great. I quickly realised that someone else had already done the parser logic work for me though! So, using the DomainHealthChecker powershell module, I scanned all these 'mailto:' domains, 14 of which had errors of their own (pretty much "DNS record doesn't exist, yo!") leaving us with 10330 domains checked for SPF and DMARC.

Results

Here are the results - note that there is some overlap, some records matched multiple fail states. There were also some failures during scanning, not sure why. Anyway, the important thing is that the total won't match up to the scanned domains.

SPF

Generally, SPF was pretty decent! 67.4% are sufficiently strict (-all), with a further 26% being moderately okay (~all) but there's still work to be done. I haven't resolved or parsed individual mechanisms to ensure they're resolvable or formatted correctly (for example, misspelt domain names, incorrect IPs, no valid A or mx record on a domain, etc)

450+ characters: 15 - 450 characters is the max for SPF records, these have more than that and are thus considered invalid
No SPF record: 437 - zilch, nada, nothing
Multiple records: 123 - Only one SPF TXT record should exist on a domain
No qualifier (~all or -all): 18 - the end qualifier of an SPF record determines what happens to all mail that hasn't matched - if this isn't specified, everything is allowed
Broken formatting: 24
- No space delimeter before an ip4 mechanism (regex: (?<![ +])ip4): 10 - Example: v=spf1 a mx ip4:255.0.1.92ip4:255.255.255.1 -all
- No space delimeter before an include mechanism (regex: (?<![ +-])include:): 13 - Example: v=spf1 a mx include:spf1.example.tldinclude:spf2.example.tld -all
- Invalid qualifier (regex: (?<![+-~])all"): 1 (this was *all lol?)
Accidental permit all: 2 - not using a qualifier (- or ~) immediately prior to all at the end of the record implies +all - I assume this was an accident
Policy is not sufficiently strict: 2691 - This is just using ~all (soft fail) instead of -all (hard fail) and is a decent enough setting. Hard fails are strongly encouraged however
Policy is sufficiently strict: 6979 - 🎉 these are good! Formatted correctly, hard failing SPF records. Just what email needs!

DMARC

Oh boy. Just over 31% of domains have something decent going on that accounts for both domains and subdomains. Only 9% of primary school domains reject on both the domain and subdomain front. Come on, this isn't good enough! From worst to best (imo):

Does not have a DMARC record: 5542 - No record at all
Domain has a DMARC record but the policy is set to none (p=none): 1412 - They've gone to the effort of setting up a record, but it was pointless because it doesn't do anything
Domain has a DMARC record but the policy is set to none for both the domain (p=none) and subdomains (sp=none): 259 - As above they've gone to the effort of setting up a record, but it was extra pointless because they've gone above and beyond to ensure that it doesn't do anything for the domain or the subdomain 🤦
0% rule: 5 - "pct=0" set, meaning the DMARC policy applies to 0% of emails. Just... Why?!
Domain is set to quarantine, but subdomains are explicitly set to "sp=none": 53 - Understandable config whilst testing - quarantine emails from the domain but maybe they're not ready to get the subdomains sorted yet
Domain is set to reject, subdomains explicitly set to "sp=none": 3 - As above, understandable during deployment
Corrupted record: 9 - Note that only the first rule results in a broken DMARC record, the other two may just prevent email report delivery
- Invalid version (v=dmarc;): 1
- Invalid rua (regex: rua=(?![mailto:| ])): 5
- Invalid ruf (regex: ruf=(?![mailto:| ])): 3
Domain is set to p=quarantine: 2130 - emails purpoting to come from the from: domain but from a non-authorised source will be delivered but likely marked as spam. Subdomains inherit this
Domain and subdomain explicitly set to quarantine: 206 - As above, except subdomains have been explicitly configured
Domain is set to reject, subdomains set to quantine: 38
DMARC set to reject domain (subdomains inherit this) properly: 551 - 🎉 This is ideal!
DMARC set to reject domain and subdomains (explicitly set 'sp=reject') properly:395 - 🎉 This is also ideal!

Closing

So yeah as expected DMARC is pretty terrible, though SPF was better than I expected. There's clearly work to be done on this front. I wonder what that state of secondaries, colleges and universities is... hopefully better? It'd be pretty easy for me to find out now I have all these scripts sitting here...

I have the data if anyone wants it, just let me know. Though to be honest it's pretty easy to generate it yourself.

If you want to check SPF, DMARC, DKIM, and other related things on your (or anyones) domain yourself, check out mxtoolbox.com.

Wait hold up, why no DKIM?!

Determining the DKIM subdomain isn't really feasible without recieving an email from the domain in question, and I'm not about to spam all these domains hoping for a reply (well, a more sensible idea will be to email a junk mailbox and hope for an undeliverable reply, iirc that will have the required info in the message headers). You can probably guess a good 50-80% of them maybe, but again I'm not going to be the source of abuse, particularly when it comes to DNS. I've seen some random domain key subdomains when looking into this, and in the past. Things like ns427365412._domainkey.domain.tld so good luck brute forcing those.

Prioritising Many Projects

2025-02-18

Prioritising Many Projects

Posted on Tuesday 18th of February 2025 at 22:28

I've got this nifty little system of organising a large number of different projects, that all have to be done, into a particular order depending on what benefit I'm trying to achieve through the projects or my constraints at the time, and it uses alphabetical ordering to organise it so it can happen in a spreadsheet (instead of putting a number in front of a word so it appears at the top of the list when you order the column, instead of half way down it) - It's not anything new but I've found this minor frustration with all the tools I've used to organise things, so I instead opted to do what all nerds do, and made a spreadsheet that did it for me in the way I needed.

Disclaimer: Whilst I actively use this in a professional setting, I may have made a severe logical error somewhere, which is mostly why I'm writing this up here. If you spot a mistake, an improvement, can think of a funky name for this, or preferably point me at someone who has done this already and better than I have here so I can just use their system instead... let me know!

If you only want to read about this organisational spreadsheet thingy, skip ahead to the Prioritising Many Projects section. If you just want to see an example spreadsheet, here's a demo .ODS file (works with LibreOffice Calc and probably Excel?). Otherwise keep reading to see how I organise planned and unplanned work, why I needed something a little different, and how it came to be.

How I Prioritise and Organise Planned and Unplanned Work

I feel like I need to start at the beginning here and outline how I already prioritised my workload so you can better understand why I needed something else, later.

I use a couple of different methods of prioritising and managing workload in the day job. That work comes in several different forms; It could mean reactive work tasks which are typically unpredictable, individual, isolated problems that are blocking or slowing a task and need fixing fairly quickly - think things like support ticket escalations - or proactive work, which include planning a project, executing a project, doing routine upgrades, etc. The individual tasks within this proactive work are generally one step in a larger project which, when most-if-not-all are completed, achieves a goal. Think "migrate service" or "upgrade component" or "launch application" as the goal, with all the steps between now and the goal being achieved as tasks. Pretty typical stuff found almost everywhere in tech and project-oriented work.

For myself, a projects tasks are managed with a Kanban board. I generally, though not always, structure these boards with six or seven columns. Within the columns are the individual tasks, prioritised top to bottom with, essentially, most important or pressing at the top of the column, moving down to least important (or tasks I can't do until other tasks are done.) Okay okay, fine, I'll go into more detail: These columns, for me, are typically something like the following:

Needs - This project, or the organisation, needs these things to succeed
Wants - This project, or the org, would in some way benefit from these things being done but they don't need to be done
Blocked - This is work that is in progress but is blocked by something out of my control (like, waiting for a part to be delivered, data from a third party, etc)
In Progress - This is what I'm actively working on right now, limited to one or two things because you can't literally do more than one thing at a time. Well, I can't multitask (effectively) anyway
Cooling off - This task may be complete but perhaps the change is rolling out, or the task has been worked on but is in trial/testing phase and I'm waiting for feedback. Maybe a change is being scream tested and just needs to be parked somewhere that isn't In Progress or Blocked.
Needs Review - This task is complete in my opinion but needs to be, or should be, reviewed before being designated as Done. Not all tasks go here, some skip over to Done, but many do because maybe I messed up somewhere! Documentation updates, config changes, etc. Anything that should have a change control process around it is double-checked
Done - These are all the things I've completed (and have been signed off) so far

Pretty typical, really. I do change it up occasionally, depending on personal preference, for a project I own. Sometimes I don't use both the Cooling off or Needs Review columns, it really depends on the project. Rather than using swimlanes for projects, I spin up a new kanban board. Sometimes I'll swap out the Needs and Wants columns for a single Backlog column. This structure has served me well for the planned work stuff, including the day to day sysadmin type things I get up to.

For the unplanned stuff that lands on the todo list, but isn't a task that's part of a larger project (think emails I need to action, or break-fix support ticket escalations) I use the classic Eisenhower Matrix. Each item (ticket, job, walk-in, email, etc) is recorded and given one of four states:

Do - This stuff needs to happen now and is more important than the below categories
Decide - This stuff needs to happen, but not now if there's a 'Do' item. Decide when it needs to happen by, and schedule it (or do it right away.) This may form a task or may spawn a whole project.
Delegate - This stuff, whilst not important, should happen. But someone else could or should do it
Drop - This isn't important or urgent, so it shouldn't take up any energy. Drop it unless there's nothing else going on. (spoiler: there's always something else going on)

Again, pretty typical.

These work management 'templates' are both simple and easy to maintain and used widely, so anyone stepping into my shoes can easily see where I'm at and where I'm going. Taking the time to manage these lists and make sure they're all organised and well written and prioritised in a suitable order within the columns/groups they reside in allows me to direct energy at the things that matter the most - solving actual problems in an order that benefits others the most. I have been way more effective since finding my way into these methods and wherever I can I encourage others to find the system(s) that works for them.

The Project Tsunami

But what if you need to prioritise a whole load of entire projects? Recently, due to a few different events at work occurring at around the same time (primarily a slight shift in direction by C-Levels combined with various government guidance and regulation documents coming down the chain) I found myself with a pretty big list of projects to do. Once I had parsed the documentation (which is a government-issued document and essentially to be interpreted as law now) I found myself with hundreds of individual projects. Literally talking about over 600 projects here. Now, some of these are very small (write a document) with plenty of overlap between the projects, and others are likely formed of multiple related but individual projects which on their own are likely to span a year of effort.

I watched this list grow as I pulled out the key takeaways from the guidance and began to feel a sense of trepidation. For some of this work, I'll be going solo. For most, it'll be me and one colleague, and for the small remaining tasks it can be spread across three to four others, depending on requirements. I'll have help here and there, but... this kind of work should have a large team behind it to pull it off in a sensible timeframe.

Such a team will not magic itself into existence, so I had to wrap my head around how to organise this work and complete it in the most effective way. How could I deliver the important projects? How could I even decide what was more important in such a huge list of projects?

I plugged the projects into my existing system, and whilst they worked to some extent they couldn't quite show me what I wanted. They would allow me to see the most critical tasks or the ones I could probably forget about (for a couple of years) but... not which ones would be cost effective, or which ones would take the longest, or which ones would have the most positive impact, be cheap (or free, financially) and not take much time at all. I tried a bunch of different methods, even resorting to an LLM to tell me about any other projects prioritisation systems I didn't know about... but couldn't find anything that fit well enough to my liking. So I got thinking... What if I took it all back to basics, used the Eisenhower Matrix and customised it a little?

This is nothing special or revolutionary! The Eisenhower Matrix is simple exactly so that it can be customised. I sat on it for a few days, thought about it, played with some ideas and was finally hit with the inspiration I needed. The next day I marched into the office with a plan and put together the following.

Prioritising Many Projects with... uhh... CDPQSW? IUECP? Or whatever it'll be called when either 1) I can think of something clever, or more likely 2) when someone tells me this exists already and everyone knew about it how did you not know about this wow how do you even manage to survive a day

Here's a demo .ODS file you can take a look at! It's made with LibreOffice Calc but I think it should work in Excel too. The formula absolutely works in Google Sheets, though I haven't attempted to import this particular spreadsheet to verify it remains sensible.

By organising your list of projects in the following way, you'll be able to highlight which ones are the most critical and cheapest and easiest, highlight if there are any quick wins, and all in a way that makes them stupidly easy to sort and rearrange.

We start with an empty spreadsheet - in column A, put Project as the header in row 1, then in row 2, 3, etc write down your many projects in any order you like. In columns B to E we rate each one on the Importance, the Urgency, the Effort and the Cost. Put those four categories as headers in row 1 of each column, then use Data Validation (as per Google Sheets parlance) to create a dropdown box for every cell in that column (except for the heading) with the following options:

Importance - How much of a positive impact will this have when complete?
- Zero
- Small
- Medium
- Large

Urgency - How much of a negative impact will this have if you don't do it?
- Zero
- Small
- Medium
- Large
- Duty
  (A Duty is something that you absolutely must do - perhaps it's a meeting with HR or a legal requirement like PCI-DSS. You might want to skip it, it might not be useful, it might not even have any negative impact if you skip it, but tough. Ya gotta do it.)

Effort - How much effort (time) will this take to complete?
- Zero
- Small
- Medium
- Large

Cost - How much money is needed to get this complete? Licensing, parts, contractors. Not employee wages for me, but maybe you should consider that.
- Zero
- Small
- Medium
- Large

In column F, put Priority as the heading. Make the data a filter or a table, then in cell F2 paste the following formula (tested on Google Sheets & LibreOffice Calc):

=if(AND(B2="Large",C2="Large"),"Critical", if(AND(C2="Duty"),"Duty", if(AND(C2="Large"),"Pressing", if(AND(OR(D2="Zero",D2="Small"),E2="Zero",OR(B2="Medium",B2="Large")),"Quick Win", if(AND(OR(B2="Small",B2="Zero"),OR(C2="Small",C2="Zero")),"Withhold","Schedule")))))

This simple but pretty lengthy formula is a basic series of IF statements that determine the priority of a project for you. What's especially neat about the words used in the B-E columns (zero, small, medium, large, duty) and the words generated via this formula in column F (critical, duty, pressing, quick win, schedule, withhold) is that their alphabetical order is also the order of priority. No mucking about with complex formula or adding 1-, 2- to each phrase, they're entirely human readable and descriptive and instructive!

Critical - Negative consequences if not done, positive consequences if done. This is essentially a disaster and requires immediate attention - think: production systems offline, threat to life/health,
Duty - You are obligated to complete this ASAP (eg: meetings, legal requirement, etc) regardless of positive or negative consequences
Pressing - Negative consequences if not done
Quick win - Easy and free tasks that you can smash out quickly that have a positive consequence. This could be condensed into the Schedule priority but I liked having it separate and clearly defined to easily pluck them out in the gaps between other bigger more stressful projects
Schedule - Positive consequences if done, these should be organised for the future in a flexible way
Withhold - Low or no consequences whether done or not, probably not worth considering yet

You'll need to then drag that formula down to populate each cell in the F column, for each row you have a project in. Then the magic happens - you can now sort the spreadsheet in a couple of different ways to order the projects how you wish to tackle them. Here are some examples - whilst it's a bit awkward to sort things ascending and then another column descending or whatever (depending on what you want) it comes naturally once it clicks:

1) Highest priority, 2) easiest, 3) cheapest

I use this personally - doing the highest priority jobs first, with the least time consuming options first, descending in financial cost order within those effort bands. Sort them like this:

Cost, Z-A (Descending)
Effort , Z-A (Descending)
Priority, A-Z (Ascending)

1) Highest priority, 2) cheapest, 3) least effort

Subtly different, but sometimes you may want to get the highest priority, least expensive jobs out of the way first even if they require more effort. Just sort it like so:

Effort, Z-A (Descending)
Cost, Z-A (Descending)
Priority, A-Z (Ascending)

1) Biggest positive outcome, 2) cheapest

Sort your projects by the ones that have the biggest positive outcomes, doing the cheapest ones within each Impact band first.

Cost, Z-A (Descending)
Impact, A-Z (Ascending)

1) Most time consuming 2) Most expensive

These will probably be the most difficult projects, you might want to do those first, regardless of the impact or urgency:

Cost, A-Z (Ascending)
Effort, A-Z (Ascending)

And many more...

You can play about with the sorting across the columns in a few ways to generate the order you want to make use of, then just start at the top and work down. When you sort things, just figure out how you want them to be ordered, then sort the columns in the apprpriate ascending/descending way in reverse order.

It's simple and nothing new, but I am real pleased with the whole "alphabetical precedence" of it. I also like how I can easily plug things in to this format. Even tasks within a project can be prioritised this way (though I prefer my kanban boards still) as it scales down to the task level pretty well.

If you find yourself with a massive workload and you have no idea where to begin, there are many systems out there that'll help you figure out which order to tackle the work in. Alternatively, if you're feeling brave, give this one a whirl!

Website Carbon

2025-02-13

Website Carbon

Posted on Thursday 13th of February 2025 at 22:10

Earlier today I saw a link to websitecarbon.com. I can't find the toot that linked to this website, despite only finding it a few hours ago (sigh) so apologies to that person!

I chucked this very website through it and, happily, it reports:

Hurrah! This web page achieves a carbon rating of A+ - This is cleaner than 100 % of all web pages globally

There are more stats on the page if you click through that put some perspective on that rating, as well as a description of how they come to that conclusion. Of course, they can't know what happens on the backend. For all they know I could be mining crypto on here. What is more likely is that my functional backend code is terribly inefficient... (disclaimer: I don't mine crypto :D)

But it's nice knowing that, at least from their end, it appears as though I'm doing everything efficiently! If you have your own website, have a go at getting it analyzed, how do you fare? Are there any ways you can improve things? Write about it and/or let me know. Every little bit of recurring efficiency you can eek out of a system is good for the Earth!

I have plans to self-host this on my own (low power) equipment in the future. Then I'll truly know my impact on the environment as it pertains to hosting this site, and can more effectively enhance things myself.

Soon™.

Chickens

2025-02-10

Chickens

Posted on Monday 10th of February 2025 at 15:02

We had to look after 10 three-day-old chicks over the weekend. Initially I wasn't too fussed one way or the other about this endeavour, but one look into their sleepy fluffy little eyes and I was smitten. It wasn't long before I had them all sleeping on me as I lounged on the sofa.

I've become quite attached to the smallest of the flock, the last one to hatch, who has taken to hopping onto my hand and nuzzling into my neck. My wife has become attached to one of the first to hatch, too, who also hops into her hand right away.

We have liked the idea of looking after rescue chickens for a while, as pets, but with all the work that has been needed in the garden we've not been able to provide a suitable home. That work, however, is now complete.

So I guess it's time for chickens?

We'll only really be able to have two or three, and there's a lot of work to do before we have somewhere for them to live outside. We have borrowed a home for inside already, and we'll have a couple of months to sort outside accomodations and security out, but we'll get there.

B'kawk!

Passion Projects

2025-02-09

Passion Projects

Posted on Sunday 9th of February 2025 at 22:10

I was lucky enough to watch the Hamilton musical in the theatre yesterday. It is, as you may have heard, incredible. I am not a patron of the theatre at all, I claim no expertise or knowledge of the goings-on, the technicalities, or the history. So my opinion ("that was an amazing experience, it blew me away in every aspect and was one of my favourite experiences of all time") does not hold much value for others. Besides, many individuals have written about the show with words and passion and knowledge that I couldn't bring to the table, so I will leave my opinion in the previous sentence and simply recommend that you watch it if you haven't and get the chance.

After the show, my wife and I had a meal somewhere fancy and discussed it, as you do. During this, my wife reminded me of the Always Sunny podcast, in particular the episode where Lin-Manuel Miranda (who wrote Hamilton and played the lead role in the USA) is a guest. We agreed we'd listen to it, and so we did. We took a slow train back home and spent the journey with the podcast playing over our earphones.

In this episode Lin-Manuel makes a particular comment about inspiration:

The thing is finding the idea that pursues you.

I loved this - I need to find the thing that pursues me, the idea that chases my every moment. I have a bunch of ideas, some of which I think could be pretty neat if I see them through. But none of them hound me for attention, none of them refuse to leave me, consume my waking hours and prevent sleep. How do you find this? I have no idea, but I suspect it involves making time and trying stuff.

The Afternoon Of Bedtime

2024-10-22

The Afternoon Of Bedtime

Posted on Tuesday 22nd of October 2024 at 21:12

Kids say the funniest things, and some really interesting words and phrases fall out of them sometimes. Their developing language skills means that they may not have the vocabulary to communicate an idea or situation, so they must find an alternative way to convey it.

Us: Did you sleep okay?

Kid: Yeah but I woke up a few times.

Us: What time did your clock say when you woke up?

Kid: It was five hundred and something (5am, digital clock reads 05:00, so 500)

Us: Oh that's early!

Kid: No it was five hundred so it was in the afternoon of bedtime, not the morning of bedtime.

The afternoon of bedtime! I love this so much. The day has a morning, at the start, and an afternoon, which comes later. So... sleep must also have a morning and afternoon! We just can't correct him on this, we don't want to. Not yet. It's too cute!

Morning of bedtime and afternoon of bedtime now feature regularly in our vocabulary, along with 't'kat' and 'pazzap'.

Theme: Paper

2024-09-09

Theme: Paper

Posted on Monday 9th of September 2024 at 22:20

I've been browsing the web looking for some cool or interesting blogs and web gardens (pro tip: 32-bit cafe) and have seen a very eclectic mix of site designs, from the retro to the chaotic and back across to the simple.

These simple ones have given me some inspiration for here, and from that inspiration comes the Paper theme - it's minimal, clean, tidy, and there's a night version too for us dark-mode fans!

That's about it. Check it out! I'm tempted to make it the default theme...

Indieweb Vs Indie Web

2024-08-28

Indieweb Vs Indie Web

Posted on Wednesday 28th of August 2024 at 22:34

Last updated on Monday 9th of September 2024 at 22:39

Indie:

One, such as a studio or producer, that is unaffiliated with a larger or more commercial organization.

-The American Heritage® Dictionary of the English Language, 5th Edition

Many people have been discussing the IndieWeb with regards to the technical knowledge and skills that are (almost) required to get involved. I first read about this in a post by @starbreaker titled "Has the IndieWeb Become Irrelevant?" which is itself a reflection on other posts and comments. It's a good post and has many valid points. In fact, I agree pretty much entirely with the underlying message, and I say that as someone sorta-involved in the community who is pondering adding IndieWeb integrations to this very website. The IndieWeb, as it stands, is a bit... perhaps elitist is the wrong word. It's a club for those that can, and although part of the IndieWeb's broader message is about eschewing the commercial web and owning your stuff, unless you know at least a little bit about websites and how they work, you're gonna be relying on commercial organisations to get you going and keep you rolling in the IndieWeb.

And you know what? I think that's okay. And the IndieWeb community and organisers know it too.

The IndieWeb chat/discord highlights this, with several prominent rooms/channels being developer oriented, but their website says it best: [...] a community of independent and personal websites connected by open standards [...]. Emphasis mine. But it's right there - "Connected by open standards." To build an independent website that connects to other independent websites via open standards literally requires programming knowledge. Sure, there are Wordpress plugins you can install and libraries you can import... You could argue, are sites that use wordpress, or plugins, or third party libraries, truly independent if they rely on these third party resources? The elitism is kinda the point. The IndieWeb is for developers, aspiring or experienced, to build something and connect with others.

"IndieWeb" however, has become the term to describe... well, the non-corporate or non-commercial web. This is sorta unfortunate in my opinion. There are other aspects to the non-commerical web that aren't IndieWeb™, this particular flavour of it just happens to have become one of the more widely known and consciously adopted words to describe this kind of thing.

There are other phrases, and in fact whilst doing a little research for this post I have come to find that there are loads of other words and phrases that describe portions of the non-commercial web and in fact align with the anti-commercial web. You've got The Slow Web which decries the instant, live-happening-now, syncronous web where reactions are immediate and the doomscroll never ends, like! subscribe! swipe left! ...and instead thrives in a slower paced world where things can wait a while and be ingested by humans at a leisurely human-healthy pace, and The Small Web & The Smol Web, where huge files and staggeringly complex and convoluted dependency chains are swapped out for standards-compliant, small, efficient and fast websites that do just enough and no more. Each of these examples encourage doing things your way, which isn't really possible on post-myspace social media. There are many more phrases, each dealing with more and more niche areas, some of which can be found on diagram.website (which I think you should check out.)

It's unfortunate that the IndieWeb™ has that name, because the perfect phrase that instils the "own your stuff, show your personality" feel of the communities represented in these discussions is, in my opinion, "the indie web". Hence the definition at the start of this article.

Perhaps some other name or descriptive phrase needs to be concocted or promoted more heavily so that the meaning of IndieWeb™ can be corrected in peoples minds. Something like "the human web" or "people net", I don't know, I'm not very good at naming things.

So the human web, the people net, the your-net. Whatever it is called, it doesn't matter. The important thing is that it is yours, if you want it. If you're tired of the conglomerate-net, disgusted by the commercialised web, sick of being the product, allergic to The Algorithm, then you can have something else. Something of your own.

You want to upload your artwork? Write fanfic? World build? Document your developing Sistrum-playing skills? Discuss your experiences slice-of-life style? Experiment with poetry?

Do it.

Use wordpress if you want. Use Blogger. Hell, use Frontpage 98 if you want. Or learn some HTML And CSS and type it all up in notepad.exe. Or just HTML, don't even bother with the CSS. Just make it yours.

Once you've got something started, you can think about registering a domain name (but you don't have to) and hosting it somewhere you have more control over (again, you don't have to) - you don't even need to pay for any of this, depending on how you decide to turn your brain-energy into words on a screen, because there are a bunch of free hosts out there. You don't need IndieWeb integrations to be part of the indie web, but if you want them, they're there for you to use.

Join us. It's kinda fun.

The only thing I will say - and I acknowledge that this does require some level of customisation and technical knowledge as the landscape of web publishing tools is nowhere near where it should be on this front - is to try and make it as accessible as you can. Not everyone navigates with their hands or consumes website content with their eyeballs. I still have work to do on this front myself.

Update: Starbreaker Reply

Posted on 2024-09-09

@starbreaker updated their post in response to this post, as well as some others who chimed in on the topic. You should go read it, it's good stuff and I'm thankful that he took the time to do this.

I wanted to put a little update out here right away but... *points at life* - anyway, here it is. Really I only wanted to highlight one part:

I don't particularly like the fact that we seem to need so many names for what I think ought to be the default. In my opinion, sites like this one, fyr.io, and even Neocities and omg.lol are the World Wide Web. Sites like Facebook, Amazon, YouTube, LinkedIn, and Twitter are the corporate Web. They're parasitic abberations, walled gardens gentrifying the internet, and an attempt by business people to turn the internet into cable TV with a comments section. We should not accept the premise that they are what the web is or should be.

-@starbreaker of starbreaker.org

Yet again, Starbreaker brings forth into the universe the words that encapsulate and clarify a feeling I've had for years and distills it into an opinion that I have fully taken on board and, indeed, share:

We shouldn't have to call 'The Indie Web' the indie web, or the small web, or any subset of the web. It's just the web. The World Wide Web. Within this WWW we've got everything, all the opinions and diversity and markets and fundraisers, charities, poetry, artwork. And, yes, a small subset of which is the corporate web (though it is small, it, by its addictive and malicious behaviour, has a very large userbase-slash-product (because you are the product they're selling, remember!)). And in and around there you've also got a sea of bots and LLM slop. But there are still whole continents of human out there, and the only sure-fire way to stem the tide of shit is to be your own island of you.

So I say again, if you want to keep the WWW healthy, you must tend to it. Make a website. Put your piece out there. Even if, like me, you feel like you've got nothing of value to say, you have.

...but maybe don't use Frontpage 98. It's terrible.

The Joy Of The Internet Slow Lane

2024-08-21

The Joy Of The Internet Slow Lane

Posted on Wednesday 21st of August 2024 at 15:58

As I write this (three days prior to the post date) I'm enjoying a quiet evening as the family sleeps, and I have barely any connection to the internet. We're talking almost zero signal strength 2G Edge, with the rare and elusive 3G HSPA, at best, on a phone. And it's kinda great?

We've taken a last minute short break away for the weekend and whilst there is WiFi available, I neither trust it nor do I want to pay an obscene amount for it. Besides, I haven't got any technology aside from the phone and had no plans to use any anyway.

I have found myself being very intentional about the websites I check and how often I check them whilst here. I'm fairly minimal on that front anyway with no social media (discord and mastodon aside, and even then they're new accounts with only a few people/servers/tags followed) to doomscroll nor any news or current event type sites on my regular reads list.

All I have done is taken some time to catch up on some indie/small/smol-web sites and have been reading the backlog of posts from my regular reads. When it comes to checking something on the "regular" or "commercial" web, I'm finding it very difficult due to the huge latency I'm experiencing here. More often than not sites, when they load at all, which isn't a guarantee, don't work in a usable way. This is often because they don't function at a basic level without copius amounts of JavaScript to render the page. The megabytes of libraries and dependencies have failed to download, or have timed out, or have errored expecting other resources to have loaded already when they haven't. Poor design, come on people this is web-building-101!

This is disappointing but not new. I often browse with JavaScript disabled and very often need to go through the effort of enabling it bit by bit. It's tiresome. This is slightly different, though. Because of the high latency I'm finding myself trying to navigate somewhere (for example, looking up a movie on IMDb or something) as a side quest to the main quest that is whatever I happen to be doing at the time. What I mean by that is that I type my search query into the URL bar, put my phone down and carry on with what I'm doing. When I get a chance I'll pick the phone up and see if it worked. If not, I try again (if I can still be bothered and it's actually still worthwhile) and if it has worked then I'll tap the relevant search result. Down goes the phone, back I go to whatever I was doing. I'll check again in a few minutes probably.

If this click-and-wait seems familiar then you're likely an older reader that hails from the dialup days. I remember well the frustrations of those days, but have a more recent experience of this too.

Before I moved into the current home I had FTTC, 80/20 internet. It seems about average now for a residential connection but at the time felt really fast. When we moved into our current place, the best internet we could get was a fairly high latency 2/0.2 (2mbps download, 0.2mbps upload) ADSL connection. We were literally on the very cusp of a functioning connection. We could just about stream Netflix but it wasn't great. YouTube was often okay on 360p but sometimes that wasn't thin enough either. And it would all come crashing down if one of us loaded a web page whilst a video was being streamed.

I distinctly remember, during the COVID lockdown, working from home and being disconnected from video conferencing software (like Zoom or whatever) when the family put Netflix or Youtube on. Audio-only (no video) on the conference call was generally okay, but any video feeds were choppy at best. In fact I never transmitted video as it would ramp up the latency so bad I couldn't do anything else. A great excuse for someone who hates seeing themselves on a screen :)

We eventually figured out some quick and easy wins to bring our upload speed up slightly. I remember our download hitting the mighty 3.5mbps one time. That was great.

A year or so ago we got fibre cables installed and now those problems and frustrations are a thing of the past!

I gotta say, I don't miss the latency or lack of bandwidth. Working in the IT field with those limitations affecting work performance was frustrating. But I do miss the resource management mini game we'd play every time something internet-y was needed. I miss not having instant gratification or answers to queries.

I kinda like the "look something up" thing not being the main thing I am doing, both now as I sit here on this low-tech holiday and back at home when we had poor internet connectivity. I am now only looking at my screen when I needed to, instead of staring at it for long periods of time, with the content streaming into my eyeballs and glueing my attention to that little square of glass. It feels... better.

I know this is a behavioural thing, I want to try to learn this as a habit. No doubt once I get back home I'll immediately go back to 'staring at the screen' being the only thing I'm doing, but I'm going to try and be more aware of how long I look at my phone. It's a tool, not a distraction from life.

Maybe I should artificially increase latency on my devices until it's normalised behaviour...

Post Reply: Css Style

2024-08-15

Post Reply: Css Style

Posted on Thursday 15th of August 2024 at 22:15

FlamedFury.com asks: What do you name your CSS stylesheet in small web projects? This post is a reply to that post.

Historically, I've always used style.css, kept inside the /css directory (as you can see on this site too!) but I've been wondering more recently whether this is a good idea?

I consider this site a small, but growing, project. I added theme support a while ago and made the decision to keep the themes in the same, single stylesheet instead of splitting out each theme into different files. This, in my mind, makes for less loading of resources. At least overhead-wise anyway. However, I'm only two CSS themes in and already I'm reconsidering.

I can imagine adding ten, fifteen themes here, and sure, switching between them would be super fast via a Javascript-powered class change on the <body> as the client doesn't need to transfer any data; the relevent stuff is already in the cache'd CSS file... but a whole bunch of people won't bother to switch themes and that is a decent percentage of transferred data wasted. Plus, having everything crammed into one file makes editing things somewhat unwieldy on my end.

So I'm considering breaking things up, perhaps a core structure file, followed by theme files which load in and out depending on the selected theme, each, of course, named after the theme they represent.

I'll probably still name that core file style.css though :D

Until then, I guess my style.css should technically be called styles.css as it has multiple styles in it... hrm.

Welcome To The Fediverse

2024-08-14

Welcome To The Fediverse

Posted on Wednesday 14th of August 2024 at 00:00

I've had an account on Mastodon, as well as some other Fediverse services, for a few years now. I quite like it. It reminds me of the early Twitter days, pre algorithm, when it was mostly technical-aligned folk chatting, networking (for fun) and talking tech (again, for fun).

I'm sure this will one day change and the diversity of individuals on there, which is already high, will continue to grow exponentially. I hope so, anyway. I also hope that the idea behind the fediverse survives and doesn't get extinguished, but I fear it may. Such is the evolution of an idea; it grows, cracks form, weeds take root, and then it starts falling apart. Generally this has something to do with the value of a share and priorities shifting and yadda yadda. You know how it all works. See the state of Reddit for a look at a deep-in-progress enshitification happening now.

For now, however, it's a generally nice place (and it's easy to remove the not-nice) full of interesting, vibrant folk with cool ideas and lots of hope for the Fediverse and, dare I say, the future.

In the interest of embracing the indieweb further, I have created another account dedicated solely to this site and my identity here. Don't expect anything great. Or anything at all. I've never been good at social anything, but it's a great way to connect with people and ideas, so I'm putting this little slice of me out there.

You can find me here: @fyr@indieweb.social.

Activity will be light, generally just cool stuff I find or learn about. Eventually I'll build support for WebMentions and other cool indieweb stuff and start syndicating things I post here out into the series of tubes we call the internet. But not the corporate internet. The human internet.

See ya on there.

Reader Feedback On Readability And Css Hyphens

2024-08-04

Reader Feedback On Readability And Css Hyphens

Posted on Sunday 4th of August 2024 at 00:00

Back in May, a reader(! There's at least one!) emailed me about an issue they had noticed when reading the content on this website.

Thank you, again, for letting me know about this issue, dear Reader. I appreciate it a lot!!

Essentially, I hadn't specified a max-width on the site, so those of you lucky folk with a large, high resolution monitor would see monitor-wide paragraphs stretching from the left-most edge aaaalllll the way to the right, which ain't at all easy to read.

I hadn't noticed this myself because I use an old 1920x1080 monitor from... let me look this up... oh. Wow! Okay, late 2009! ...I really need an upgrade.

But, I immediately understood the issue and excitedly replied, thanking them for the valuable input. I slapped together some quick CSS to limit the width of the content but it wasn't pretty.

In fact it was broken. Again, my crappy old 2009 monitor tricked me into benightedness. I hadn't realised that, whilst the site looked fine enough on my monitor and on smaller screens, on wider screens, everything under the menu was stuck to the almost-but-not-quite-all-the-way-left. Oops.

Yes, it has taken over two months to get there, but I have finally fixed it. Properly. And whilst I've gone a bit wider than their suggested max-width value, I am hopeful that they, and you, are now more comfortable when reading stuff on here.

I think. I mean, short of zooming out I can't test it as I haven't got a monitor younger than 15 years old, so let me know if it hasn't worked I guess!

Oh, I've also taken the opportunity to make a couple other CSS changes, one of which is minor (fixing some code block stuff) but the other I wanted to mention and perhaps, if another reader finds themselves here with the time to share their opinion, let me know what they think.

Since I (re)built this site I've used text-align: justify; for all the paragraphs. There's a demo here on the MSDN website that nicely demonstrates what it does, but for those of you who don't want to click through, it essentially forces the left side of a line of text to line up with the left-most side of the container, fit as many words in as it can (by growing or shrinking, to a limit, the spaces between the words) and then ensure the end of the right-most word is right up against the right side of the container.

This is generally fine (and it's what you see in newspapers) however every once in a while you'll notice a line of text that has very few words in it and huuuge spaces between the words that are there. This becomes a way more common and prominent occurence on smaller screens.

I recently learned of a fix for this, which is to use the relatively new hyphens CSS property, with the value auto. This tells the browser to fit in a bunch of words to a line, the same way as before, but to fit that too-long word at the end, break it up at a suitable place and stick a hyphen in there, drop to the next line, then continue that word.

The upside, for me, is that it allows me to keep justified text but without it looking ugly in places. However the obvious downside is a whole load of hyphens at the end of a line, and more so on smaller screens like phones.

Despite preferring this to plain-ol' justified text, I'm not entirely sold on it as yet. What do you think? Does it work? Is it hard to read? Should I not justify my text? Let me know.

Tomatoes And Anthropomorphising Buildings

2024-08-03

Tomatoes And Anthropomorphising Buildings

Posted on Saturday 3rd of August 2024 at 00:00

We've needed to replace our cess pool - a buried and sealed container for our sewage - for years. This has been a source of contention and stress for various reasons, but I am happy and relieved to be able to finally say: it is done.

What this has allowed us to do is concentrate somewhat on making the garden less exploded-bomb-dead-plants and more things-are-alive-garden. We would use it as a bit of a dumping ground and not care for its appearance, due to the very destructive work we knew would happen at some point in the future. However! That has now happened, and whilst we have very little grass left alive by the diggers, dump trucks and other large vehicles (we eagerly await the grass seed to grow) we have finally been able to commit some time and energy into the garden.

The Royal We, that is, because in reality it's been about 77.41% my wife and 22% the kid. I have dug up some very survival-oriented weeds, but that's about it.

We now have an almost-free-of-trash garden! Including neater edges, an extra metre of space along a fence thanks to my valiant weed-removal efforts, and a little pond to attract some water-based nature! We still have a long way to go, but we noticed already that it has changed from a somewhat disappointing, disgusting and disused area to one that is relatively clean and, most importantly, embodies hope of what it can one day become.

Due to the ever-forward-looking spirit of hope for the future combined with the knowledge that these big groundworks should be finished by summer, in the spring we (royal, of course) planted some fruit and vegetables. It's something we attempt most years. Perhaps, historically, the plants pick up on our dejected nature when in the garden, or maybe we just subconsciously put less effort in, because in previous years we haven't had much luck at all. This year, however, is different.

We have lettuce, cucumbers, strawberries, sugar snap peas, peppers, potatos, carrots (though not many) and tomatoes. And oh my, the tomatoes. These tomato plants, of which there are about a dozen, have all far outgrown the greenhouse within which they reside. They're so big that we couldn't get them out of the door without breaking them, so there they stay. Whilst only a single one of the fruits has riped enough to eat, we have... I'm going to conservatively say hundreds of tomatoes growing. About 10 or so are not-green, and I can't wait to eat them. As is tradition, that first ripe tomato went to the kid, who has been very helpful indeed with the planting, watering, and general caretaking.

Whilst we were clearing the garden after the week-long installation of the new water treatment plant, my wife had a thought which has stuck with me since she voiced it.

As we were cleaning the windows and soffit of dirt, dust and half a decade of accumulated detritus she raised the point that we had sort of neglected the outward appearance of the house over the last few years. With the destructive work we knew we needed, this is understandable, but now that the work is done, the home deserves some love. She described the home as being like an old being who needed some looking after, who needed some help keeping tidy. She anthropomorphised the house. And it immediately changed my perspective of what I was doing. I didn't really want to clean the soffit, and then... I did. So I cleaned them. All of them. And we've been out there every day feasible (insert general grumbling about climate change and UK weather here) doing things. Digging, removing, cleaning. Fixing. Polishing. Making the house respected, after it being neglected for decades by the previous owner and sort-of-neglected-outside by us. I see a future, not too distant, where we're actually proud of it.

So now, whenever I think about the house, it's from the perspective of taking care of a being that, by the nature of it being a building without intelligence, limbs, nor opposable thumbs, cannot look after itself. This is just my way of tricking myself into turning our home from a building site we live in into something to be proud of, and I'm okay with that.

It's gonna take a while. We'll get there. And we'll eat so many damn tomatoes on the way.

Windows93 Net

2024-07-26

Windows93 Net

Posted on Friday 26th of July 2024 at 00:00

I just saw the windows93.net website on the Jordan Eldredge site - It looks like it's been around since 2019 but this is the first time I've seen it. It's a fun site, check it out! There's loads of functional (and in some cases, intentionally dysfunctional) features and hidden surprises.

I love little passion projects like this. Things done for fun. This is what the web needs to give it some humanity.

The Month Of Illness

2024-07-25

The Month Of Illness

Posted on Thursday 25th of July 2024 at 00:00

Hey, been a while, but what a month.

I've been unable to commit time to this site over the last four/five weeks and my list of things I want to add and write is only growing; On and off illness including a throat that was so sore it hurt to breathe, the installation of a water treatment plant, my first visit to a theatre to see a stage production, and the chaos of work.

Luckily we avoided the Crowdstrike escapade. If we had been hit by a similar outage at work I'm not sure we would have done much recovery at all by now as I and the family have been suffering with a chain of illnesses that have been almost completely debilitating. Somehow we managed to be bed bound on different days so parenting could continue, though it was a struggle. And it wasn't even anything exotic that took us out - we're talking cold, flu and covid.

Let's hope the summer break limits the bugs and illnesses that get transmitted and end up knocking us down. If we're luckily our immune systems will somewhat recover by September. In the meantime, I need to update /now and /ideas.

Css Naked Day

2024-06-14

Css Naked Day

Posted on Friday 14th of June 2024 at 00:00

Last updated on Thursday 9th of April 2026 at 23:14

2026 update: Yep, we're here for 2026 too! One year since building this automation and it's kicked off itself nicely. Perfect!

There exists a day for everything, but this is one I can get behind: CSS Naked day. It celebrates and promotes web standards, specifically that a site should work with just the HTML. It falls on the 9th April. I heard about it this year on that very day as a few websites I check had implemented the idea. This website hadn't been built yet, it was in its very early stages, and I didn't get an opportunity to add it to the old wordpress site in time. I did however add it to the /ideas page.

And I can now tick it off as completed on that page, as I have now added it! I'm not late for 2024, I'm early for 2025!

On (and around, because timezones exist) the 9th of April, this site will not have any CSS (and I already don't use any JavaScript). As it stands right now you won't be able to turn it on during that window of time, but I'll probably add that feature. Probably.

The reason for doing this is clearly defined on the official site linked above, but to reiterate, it's to promote (and showcase) web standards. Ideally, all sites should be as accessible as possible and this starts with having your site be fully functional with just the HTML. How your browser or screen reader renders it should be pretty much consistent within the family of technology you are using, but if you start to add Javascript gunk or do funky reorganising of structure via CSS, some of your visitors are gonna have a bad time.

Because this feature is being added ~~incredibly late~~ astonishingly early, I have also made it a theme you can use whenever you wish. So, if you don't want to wait until the 9th of April, or if you just wanna check that I'm doing HTML correctly, head over to the theme page and enable the CSS Naked Day theme.

Then Pages

2024-06-04

Then Pages

Posted on Tuesday 4th of June 2024 at 00:00

I have been making use of the /now page, which is honestly something I thought I would end up dropping. I may only update it a few times a month but I've been fairly consistent with it.

Ever since reading about the idea on a few small web sites (and in particular browsing a bunch of them on nownownow.com) I wanted to have one, and I am glad it's turning out alright. However I quickly came to the realisation that I didn't want to delete anything...

So I didn't. I archived the file and paired down the original at the start of the month because I wanted a way to go back in time, to see what I was doing back then.

Hence, /then!

The /then page is as simple as that. It's just a monthly archive of my /now page. I'll append to the "live" /now page until the end of the month, at which point on the next update I'll archive it to /then, strip back all the non-continuous stuff (like reading a massive web serial for years on end) and resume appending the live /now page. Another thing checked off the /ideas page!

Lovely. Hey, if you haven't got a /now page, set one up. Then (or if you have one already) why not set up your own /then page to archive them too?

You know you want to.

Themes

2024-05-21

Themes

Posted on Tuesday 21st of May 2024 at 00:00

I saw the aurora a little bit ago and when I posted the more colourful pictures on here I thought at the time that the colours would make a cool theme.

Using that as inspiration, I built theme support into this site! Yay, another item on the /ideas list ticked off!

In the end, the theme became less aurora-y, despite retaining the name, and just a purple and colourful recolour of the default theme instead. It's not the best theme I've ever built for a website, but it's there!

As mentioned on the theme page, it stores the theme choice in a cookie for 30 days, after which point you'll need to re-set a theme. I'll probably change this to keep the cookie current if you visit this site within that 30 day window at some point.

Now I've got theme support I'm excited to build some more interesting and experimental designs - expect more soon!

Aurora Redux

2024-05-11

Aurora Redux

Posted on Saturday 11th of May 2024 at 00:00

I posted yesterdays aurora post, feeling happy to have seen a smudgy purple haze in the sky. Little did I know that outside things were getting significantly more magical, That was, until I decided to have one last peek about an hour later to see if anything had changed... and it had.

Seeing them with eyes isn't quite as impressive as seeing them through a camera lens. To my eyes they looks like slightly purple or green bands of glowing air that slowly shift and change as you watch, like very faint glowy clouds at night. Point a camera at them though and you get something much more vibrant.

These (and many, many others) were all taken on a Pixel 4a in Night Sight. Other than cropping, scaling and exporting (and whatever the craziness the Pixel does as it renders these) there has been no editing or post processing.

I think I'm going to use these in my next theme for this site. Theme selector, here we come!

Aurora

2024-05-10

Aurora

Posted on Friday 10th of May 2024 at 00:00

I've always wanted to see this.

Taken this evening in the south east of the UK. I'll be honest, I wasn't sure it was the aurora - it was a faint purple haze in the sky to my eyes - until I took some Night Sight photos, which gave some of the images a bit of definition.

As the pictures are taken on a phone, and an old one at that, using Night Sight, they're not that great. But still, I think they look awesome.

I would love to see it further north in all its glory though.

Yep And Nope Pages

2024-05-07

Yep And Nope Pages

Posted on Tuesday 7th of May 2024 at 00:00

I saw Jack Baty's nope list and just had to add my own. So I did.

Then I got thinking about maybe making a /yep list. I wrote out the /nope page and found that the content was mostly things that had a negative slant, and I wasn't sure if this was okay? But I then realised that it is okay! Life is positive and negative and everything in and around that, so why not make that /yep page and have it orient more to the positive side of things! Let balance be restored!

So I did.

The two lists are massively incomplete of course, I will be adding to them, but at the moment the /nope page is more technology oriented whereas the /yep page slides more to the behavioural and bigger picture life side of things. I'm sure there's an academic reason for that, but I do not have the knowledge to even begin picking that apart.

Anyway, there we go. /yep and /nope pages. They didn't even have time to hit the /ideas page. Make them yourself, see what flows out into each list, and let me know if you do.

Indieweb H Cards

2024-05-01

Indieweb H Cards

Posted on Wednesday 1st of May 2024 at 00:00

I've added an h-card to the homepage!

A what now?

An h-card is one type of microformat, which are just standardised ways of presenting different types of information within HTML. Think of an h-card sort of like the HTML version of a business card, but an agreed upon method of presenting, on a website, the relevant information you might want to put on a business card in a machine-readable way, whilst simultaneously fitting in with however you want your website to be structured and designed. It's easy to implement and encouraged within the IndieWeb community as a means of standardising how you present yourself on your personal website. As long as you have a bit of information about your alias, persona or identity on your site you can very easily turn it into a h-card by wrapping that info in a few different classes.

There's a whole list of properties you can use to present all different kinds of information about yourself, and whilst I've opted for a somewhat minimal subset of them there isn't really a required minimal set, so you can do as much or as little as you like. Though of course the more you include the more personalised and open your site will appear to anyone that comes across the h-card.

I've only recently learned about the indieweb and the whole thing appeals to me so much that I opted to drop wordpress in favour of building my own thing. This allows me to change things around and add features with ease, and it has brought back the fun of building websites. I look forward to making small incremental changes to this place!

Whilst I doubt I'll be going all-in on the indieweb dream (like, I doubt I'll bother autopublishing to social media from here whenever I post something, I'm just not into that) this is one of many changes I'll be making to bring this site more in line with the indieweb/smallweb ethos and as a result, hopefully bring a bit of personality to my little slice of the intarwebs.

Rss For The Win

2024-04-26

Rss For The Win

Posted on Friday 26th of April 2024 at 00:00

The old site used Wordpress, which came with RSS feed functionality built in. This is super useful, I'm a fan of RSS. It's a great, simple way to keep up to date with content posted to a site automatically without needing to visit the site and look with your eyeballs.

When I first updated this site from the previous Wordpress install to this custom... thing... I built, I didn't really bother with RSS feed functionality on day one as I honestly thought nobody was using it, however after poking at the logs I'm seeing a decent number of requests for the old feed url.

So for the first official site functionality upgrade, I have added (back) an RSS feed!

It's pretty basic right now, doing a little bit more than the minimum required by the spec, but I'll add useful and relevant functionality at a later date. I'm using the same URL as the previous Wordpress feed too, so anyone who subscribed to it in the past should resume getting updates now without changing anything on their end. At the moment it's only the title of a post and a link to it pretty much, no description or post images or whatever.

I do know though that those dear RSS feed users will get a dump of all the post links as if they're all new and unread, as the unique identifiers for the posts will be different. For that I apologise.

If you use the RSS feed and want a particular element or feature added, or if I'm doing something horribly wrong and haven't realised, let me know.

A New Website For An Old Web

2024-04-18

A New Website For An Old Web

Posted on Thursday 18th of April 2024 at 00:00

I have spent the last couple of weeks reading about the indie web, the small web and the slow web. All related concepts that I personally boil down to "what the internet used to be like, until the corporations took over". That transition was probably somewhere around 2006? I'm not sure. Before the Big Web took over the majority of user attention, people made websites for themselves or for people with similar interests, and people who wanted to read things that interested them followed links from small time website owners or used explorative services like StumbleUpon to find new content. Of course, nowadays there are way more people on the 'net, but the majority of people spend more of their time on way fewer platforms like facebook, instagram, tiktok, etc. I miss the old days of small quirky websites built for fun. For some reason, having a reliance on WYSIWYG editors and bulky CMS solutions takes the fun out of it all. For me anyway.

Anyway, a couple years ago I came across shellsharks.com. I read something interesting on there, bookmarked it, then moved on. Then I came across the site again. And again. And then finally, a few weeks ago I decided to really explore it, click around the menu and read some of the other content. What I found outside of the blogpost or list I had been reading prior was a technical/infosec site with a whole host of cool features. It made me think about my own place on the web and what I wanted it to be for me, and for anyone who found it. I dove down the rabbit holes each of the features the shellsharks.com site had, only to find that they weren't unique nor were they overly new in many cases, from the idea behind a /now page, to having a public website changelog and integrating federation services. The key was doing it all yourself, for yourself. This appeals to me.

But what really got me thinking was the idea that the internet doesn't have to be Big. It can be Small. We don't have to have huge dependancy chains, host our sites on top of a gargantuan codebase that tries to do everything (and is slow as a result) or abide by any kind of user interface design principles in a conformist "everyone-else-does-it-this-way" way. We can play. We can experiment. We can break things.

This, as well as the recent xz supply chain attack, in turn got me thinking about my site from an infrastructure perspective. What did I want from it? The answer was simple. I wanted it to be something fun (for me), something useful (for you) and something mine. I wanted to know it inside out, to tinker with it like we used to back in the good ol' days. To break things, to make things.

So that's what this is. I've ditched Wordpress - I don't have anything negative to say about it, but many of the decisions I would make about web design, primarily on the backend, were made by someone else. That's fine. But I want to make them. So I built this. This very basic site. The only dependencies it has are PHP for some very basic logic stuff and some kind of web hosting software. In fact, I develop it on PHP's built in web server so whilst that's not suitable for production it is in theory optional to have web hosting software like nginx or whatever.

What's different?

There are plenty of features I haven't added yet (like an RSS feed) but the point of this rebuild is to plug away at my leisure, adding things as I see fit. It's mostly content right now; Here's what this new version of the site is launching with:

I have ported across the most recent batch of blog posts from the old site, plus some of the more popular ones from longer ago
I have a /now page, showcasing things I am personally up to around about nowish - what I'm watching, reading, working on, etc
I have a /changelog page, highlighting changes I make to this site (on the front or back end)
I have an /ideas page, a dumping ground for what will probably be bad ideas either for this site or for just... anything really. I always forget my ideas, hopefully not anymore
I have a /wander page, showcasing a bunch of cool stuff I like on the internet that you can wander through

I aim to add an RSS feed, themes, plus a bunch of other stuff (check the aforementioned /ideas page) and eventually port over more of the content from the old site.

As for content, what happens happens. I hope that adding features and documenting them introduces a certain momentum which results in more blog posts and /tools. I'm also hoping that I can start to integrate more into the indieweb community and the fediverse, there's some great people and content out there and it all feels like the web used to.

It's kinda exciting.

Reset Sims Net Password

2024-03-03

Reset Sims Net Password

Posted on Sunday 3rd of March 2024 at 00:00

If you're trying to migrate or take over a Sims.Net install (say, from an unfriendly MSP, or you are migrating Sims away from a hosted solution to on-prem) or have just forgotten the admin password, you might find yourself with access to the database but without any access to Sims.Net itself - perhaps no passwords have been shared out of spite or the data export/import process has gone a little awry and account information hasn't pulled across quite right.

Sims.Net uses its own internal accounts for access via the application which can be quite awkward to unwind. As long as you have SA access to the database, you can recover from this - Luckily there's a built in routine you can run which makes resetting the sysman account, which will give you enough permissions to get back up and running, a breeze.

Assuming you have access to the database (if you don't have access to the sa account, read this) you can:

Open SQL Management Studio
Right click the Sims database and select New Query
Paste in the following:
exec sims.db_p_reset_sysman_password
Click Execute

Now you should be able to log into Sims.Net with sysman as the username, and password as the password. It'll ask you to reset this when you log in. All being well you should now have access to Focus > System Manager > Manager Users, and therefore able to escalate privs to take control of things.

Useless Windows Bug 5462

2023-11-08

Useless Windows Bug 5462

Posted on Wednesday 8th of November 2023 at 00:00

Here's a bug in Windows explorer that I happened to notice today, which has zero value and despite me trying to find a way to abuse it, is seemingly completely useless. Tested on Windows 10 22H2 & Server 2019/2022.

If you right click on the start menu icon in the task bar, the task bar across all screens freezes in time. The clock doesn't update, applications that open after you right click don't show up on the task bar as being open (except in Server 2022), but when you click away from the right-click menu everything snaps back to the correct state.

Right clicking on other things in the task bar doesn't yield the same results. It's only the start menu icon.

EDIT: Does not affect Windows 2012R2.

Vms Not Booting After October 2023 Windows Updates

2023-10-18

Vms Not Booting After October 2023 Windows Updates

Posted on Wednesday 18th of October 2023 at 00:00

Booting a VM and you get the error "Failed to power on with Error Incorrect Function"? Yeah, this hit us too. Looks like it hits VMs with secure boot enabled.

Quick fix: Move, delete or rename the .mrt & .rct files associated with the .vhdx file that the OS of the non-booting VM normally boots from. It'll then boot normally.

Alternatively you can uninstall the updates kb5031364 and/or kb5031361.

Migrate Sims Net Server

2023-10-12

Migrate Sims Net Server

Posted on Thursday 12th of October 2023 at 00:00

Sims.Net is a very common and, in my opinion, a very badly written Management Information System used in the UK's lower education sector - primary and secondary schools. It's commonly used not because it's good, but because it was one of the few options many years ago and a huge number of schools just haven't bothered to move to something else. And I don't blame them, it's got a lot of history and data going back decades in that database. It's not an easy thing to migrate off of.

ESS, the (current) caretakers of Sims.Net, have recently changed their licensing model to seemingly prevent third parties from hosting Sims.Net data on behalf of a school. I admit, I don't know the details, but that's the message as I understand it.

It looks like due to this, and the EOL of Server 2012R2, many school techs and sysadmins across the UK will be scrambling to get Sims.Net installed on an on-prem server.

Unfortunately, documentation is sparse. It's a shame, but it costs a whole lot of cash to pay for third parties to do this for you. Right?

In the spirit of publishing guides for stuff organisations seemingly don't want guides published for I've written up a guide on installing or migrating Sims.Net (and the horrible, horrible SOLUS3) server/database, because I struggled to find any well written and up to date guides and had to figure it out myself recently.

So if you're looking to install or migrate Sims.Net, hop in and come for a ride. It's easier than you think! Until you get to SOLUS3. We won't be migrating that. But I'll go through how to install it fresh!

This is a fairly rough stream-of-consciousness guide based on internal documentation I use to migrate Sims - suggestions, advice and corrections are welcome. I will continue to refine this over time, noting any significant changes at the bottom of the article.

Ingredients

You will need the following software to proceed, as per the SIMS Minimum Hardware and Technical Roadmap documentation:

Windows Server 2019 (with GUI)
Microsoft SQL Server 2019
- Express: This is free. Useful for primaries or small secondaries. The only reason to move to Standard will be if your database is bigger than 10GB
- Standard: You need to pay for this. The only feature we need it for is if your Sims.Net database is bigger than 10GB in size.
Sims.Net installer files (you can get these from your current SOLUS3 install by extracting the update that matches your currently running version of Sims)
SOLUS3 installer files (linked below)

Note: This guide was written and tested on 7.210, the 2023 spring release, however I can't imagine it's any different for any other Sims.Net 7 release.

Installing Windows Server 2019

Set up a fresh Windows Server 2019, ideally in a VM, to be dedicated to the task of hosting Sims.Net. Given the common configuration of networks in schools I am assuming you're also joining this server to a Windows domain.

You will need to use the GUI, for SOLUS3, unfortunately. If anyone has got this working without the GUI, let me know.

Why 2019? Well, officially, it's the latest version of Windows Server that Sims.Net officially supports. Whilst I'm certain it'll run perfectly well on 2022, if you try it and need ESS help, the first thing they may say is “wrong OS, bye!” so we're playing it safe here.

Try to keep Sims.Net and SOLUS on a server dedicated to the task and not sharing roles with other stuff, except the very few sims data extractors that absolutely must be installed on the same server as the Sims database. (Many of these data extractors are bullshitting when they tell you they require this, by the way, but that's a tale for another post.)

You'll need a good amount of storage and RAM available, but requirements vary depending on workload.

According to Teh Intarwebs, the general recommendation is to use a fixed amount of RAM with MS SQL. You should allocate RAM by getting the database size, plus ram for the OS, plus growth. So if your Database is 20GB and you expect it to grow by, say, 4GB over the next few years, throw 32GB of ram in there. That'll give you 20GB for the database, plus 4GB for DB growth, plus 8GB for the OS and any work it, or applications on it, has to do. If you're doing this in a VM you can, of course, raise this much more easily later if and when needed.

Now... this may be overkill, but... create at least two but ideally five storage drives. If you can only create two, keep the OS and docstorage on one and the database stuff on the second. This guide will go through the creation of five of these drives as and when they're needed.

I will be using the following throughout this guide:

C:\ - System - Windows and program files
D:\ - Data - Database
E:\ - Logs - SQL Logfiles
F:\ - DocStorage - file storage
G:\ - TempDB - SQL TempDB

Installing Microsoft SQL

Disclaimer: I am not a Database Admin, it's a form of advanced magic that my brain is not capable of retaining! So, if I haven't followed best practices here let me know and I'll research and correct.

Download SQL Server ~~2016 SP3. The latest version of Sims that this guide was built with (Spring 2023, 7.210) only officially supports up to 2016 SP3~~ EDIT 2023-10: Turns out this isn't accurate, the document I was using was out of date. SQL Server 2019 is supported (as per latest guidance, PDF warning) and in fact I recommend it as it will allow you to encrypt the SQL connections the Sims.Net client makes by default. SQL 2022 is still not supported, however it will be from the Spring/Autumn 2024 release. Whether you use Express (free) or Standard (which you need to pay for) depends on your usage requirements. Primarily for our use case, the major decider is how big the database will be. Typically (but verify this by checking your existing system requirements yourselves):
1. Primaries: Download SQL 2019 Express: https://www.microsoft.com/en-gb/download/confirmation.aspx?id=101064
2. Secondaries: You may be able to get away with Express if your database is below - and not expected to exceed - 10GB in size. Otherwise, buy and download SQL Server 2019 Standard Edition
Run the Express installer, or if using Standard, mount the .iso and run setup.exe
1. Express (verified with 2019):
  1. On the popup, select Custom
  2. Select Install to download and run the installer files
1. Select Installation > New SQL Server stand-alone installation...
2. If you are using the Standard installer but only need express, with its associated limits (good enough for a primary in most cases) select the Express edition in the Specify a free edition dropdown, however if you need Standard or Enterprise, select the Enter the product key radio button and, if it's not auto-populated, enter the product key then Click Next >
3. Accept the product terms and click Next >
4. On Standard, it is strongly recommended that you select the Use Microsoft Update to check for updates checkbox, then click Next > to begin checking for, downloading and extracting any updates
5. On the Install Rules section, get everything to not fail and click Next >
6. On the Feature Selection window, select Database Engine Services and if it's selected automatically, untick Machine Learning Services and Language Extensions - you can leave these features on the %SystemDrive% - then click Next >
7. Set a Named Instance name to Sims or something else descriptive for easy management later, then click Next >
8. In Server Configuration:
  1. Accounts: it's generally recommended to create basic domain accounts for the SQL Server Agent and SQL Server Database Engine services, leaving SQL Server Browser as NT Authority\Local Service, however the defaults do work fine in our use case.
    1. Grant Perform Volume Maintenance Task should be unticked for a server hosting only Sims. It makes database creation faster but can result in data disclosure.
    2. Collation: Leave as default
      1. The default is Latin1_General_CI_AS. The ci bit indicating case-insensitiveness. I haven't dug into this but I'm wondering if this is this why Sims.Net passwords aren't case sensitive. I suspect they don't hash the passwords, perhaps they encrypt them? I'm interested in trying this without the ci to see if Sims breaks and passwords become case-sensitive. Or perhaps they just LOWER() or UPPER() them or something. If you know, let me know!
    3. Click Next > once you've configured this section.
9. Database Engine Configuration:
  1. Server configuration: Select Mixed Mode and insert a strong password
    1. Don't use a space in the password. SOLUS3 will happily use a password with a space... until you come to migrate the database and all of a sudden it won't like it any more in the password field in Settings > SOLUS3 > SOLUS. Also, don't use more than 25 characters because some (but not all!) password fields you need later will only accept a max of 25 characters. And they don't tell you this. Yeah, welcome to SOLUS3.
  2. Add any windows groups/accounts that you wish to be able to perform admin tasks on the database
  3. Data Directories: Although you can if you have no choice, you should not store any data on the %SystemDrive% (C:\ generally), you should instead store data on separate drives if possible. If you're using a VM like most of us are for this, create two new .vhdx files with a 64KB allocation size and put the Data root directory and Backup Directory on one (D:\ in my case, labelled Data and about 120GB in size), moving the User database log directory off onto another (E:\ in my case, labelled Logs, 50GB in size) - if you are lucky enough to have the capacity you can store these extra .vhdx files on different physical disks, or if you're not using a VM for the Sims server you could allocate separate drives.
  4. TempDB:
    1. There is no general consensus on whether to store these files on separate disks, with data or with logs, as each situation is unique, but in the interest of squeezing out as much performance from the horribly slow Sims.Net as possible I'm opting to store them on what is now my fourth .vhdx - a 50GB w/ 64KB allocation drive I can shrink (or grow? Yikes) later as required, labelled TempDB. Whilst it's early days, I've yet to see this grow much. Currently sitting at about 10GB.
    2. Data Files:
      1. Number of files: Same as your number of logical/virtual processors, up to 8 max
      2. Initial Size & Autogrowth: This depends on the workload, but generally you want to increase these numbers from the defaults - each time the file has to grow it introduces some overhead so you don't want it to be too small, and at the same time you don't want to use up loads of space on disk that isn't being used. I'm using 1024MB for the initial size and 512MB autogrowth.
      3. Store these on a new drive (G:\ in my case) as mentioned above, not the %systemdrive% and not the data or logs drive.
    3. Log Files:
      1. Initial size & Autogrowth: 512MB/256MB
      2. Storage: Same as the Data Files - G:\
  5. Click Next > and if all the tests pass you should be able to review the install - if you're happy with the provided summary, click Install
Once install is complete, install the SQL Server Management Tools which you can do from the Installation section of the SQL Server Installation Center installer you probably still have open, or you can download the up to date version here: https://aka.ms/ssmsfullsetup
Reboot

Sims Migration & Installation

It's worth taking some time before the planned migration to audit any and all data extractor tools that pull data from Sims and sync it with another provider or database. These will all need to be touched or outright migrated, too. 9/10 times the support for the provider will do this for you, but they're generally pretty easy to move in my experience - Some, however, are not. A guide for another time, perhaps.

The migration process quickly takes Sims offline for everyone so schedule some downtime before you proceed to step 5 below. You can get up to that point beforehand however.

Create a folder in the Storage drive called Sims Shared Files and share it (Sims$), sharing with the appropriate ACL groups in place - whilst most staff probably won't need access, some will (cover, attendance, data manager, etc)
1. I suggest making three groups in AD, ACL_Sims_Read, ACL_Sims_Write, ACL_Sims_Full and giving these read, modify and full access as appropriate to the share and the folder. You can then easily give people the relevant access (read or write - Full should be reserved for sysadmins) later without making changes to the folder and files directly
Create a folder on the C:\ drive somewhere, like the desktop - it can all be deleted once we're done
Copy the installer files into this folder: SIMSApplicationSetup.exe & SIMSSQLApplicationSetup.exe
1. If you don't have these, you can extract them from SOLUS3:
  1. Open SOLUS3 on the old server
  2. Upgrades > Update Library
  3. Select the latest Full Release that matches what's running on the server and click Extract
  4. Select a location to drop the files, then copy them to the new server
Run SIMSSQLApplicationSetup.exe on the new server
1. Click Next >
2. Select Browse to change the install location to the new SQL Server Binn folder, e.g. C:\Program Files\Microsoft SQL Server\MSSQL151.SIMS\MSSQL\Binn and click Next > then Next > again
On the old server, run DBAttach as admin
1. Server Name: <current sims server hostname>\<instance name>
2. Username: sa
3. Password: <sa password>
4. Click Connect
5. In the Detach a Database section, select the relevant Sims database... Don't click Detach, just hold there for a moment... I've had issues with things using the database which prevents DBAttach from working - whilst I haven't figured out a nice way to work around this, I do have a dirty way of resolving it - take the database offline, then bring it online again. If you try to detach the database here and nothing happens for 10 minutes+, here's how to encourage it in a way that will probably make DBAs shudder:
  1. Open Microsoft SQL Server Management Studio
  2. Log in
  3. Right click the relevant Sims database and select Properties
  4. Click Files and copy the location of the database file. It'll be something like:
    D:\Administration\Server Applications\MS SQL Server 2012\MSSQL11.SIMS2012\MSSQL\DATA Keep this for later (check that the .ldf file and the .mdf file are in the same folder, if they are different take note of both locations)
  5. Close the properties, then right click the relevant Sims database and select Tasks > Take Offline - might take a while.
    1. If this doesn't work within 5 minutes, you'll likely need to stop any sync tools running on the server/database. You can execute EXEC sp_who2 on the database via Microsoft SQL Server Management Studio to see what's actively querying it
  6. Right click the same database again once it's offline, and select bring online
  7. When this has finished, quickly go back to DBAttach and click Detach - this again can take a while.
Open the data directory you grabbed earlier containing the now-detached sims database
Copy both the sims.mdf and sims.ldf files to the new server, feel free to put them in the root of the data drive or something, these copies will be deleted later
Run DBAttach on the new server as admin
1. Server Name: <new server hostname>\sims
2. Username: sa
3. Password: <sa password>
4. Click Connect
5. Under Attach a Detached Database, add Sims as the database name
6. Select the ... button and locate the database file you've copied over from the old server
7. Click Attach
  1. If you get an error here about missing files, odds are your database files are named differently (for example, sims.mdf & sims_log.ldf) - just rename them so they're both the same, the only difference being the file extension (.mdf and .ldf)
8. Once this is done, check the DB .mdf and .ldf files exist in the right directories, then you can go ahead and delete the copies you brought over from the old server
Edit the SIMS.ini file in c:\windows and add SIMSSetupsDirectory=S:\Setups to it
Run SIMSApplicationSetup.exe to install the Sims client
1. Install with the defaults
Create connect.ini in C:\Program Files (x86)\SIMS\SIMS .net and add:
[SIMSConnection] ServerName=.\sims DatabaseName=sims
1. Optionally, if you log in to Sims with windows accounts instead of Sims/SQL ones, which I advise, add ConnectionType=Trusted to the bottom of the file, too.

The Sims.Net client should now log in, on the server at least! Test it and make sure accounts and data exist as expected. If, however, you find that you cannot log in via Sims.Net itself (sometimes the database import/export goes a bit wrong, or you've not been given any account details from the previous provider) you can easily reset the built in sysman account by following this guide.

ESS recommend setting the database Compatibility level to SQL Server 2012 (110). ~~I haven't tested the difference in much depth, but a cursory anecdotal glance at this resulted in no discernable difference in performance~~ We have noticed a significant slowdown on reports involving leavers, and database locks on reports involving future students that disappears when the database compatibility level is set to SQL Server 2012 (110) so I now strongly recommend that you do this. To make this change:

On the new server, open SQL Management Studio
Expand Databases in the Object Explorer
Right click on the Sims database and select Properties
Select the Options page
Change the Compatibility level dropdown to SQL Server 2012 (110)
Click OK

DocStorage

On the new server, .NET 3.5 is required
1. Mount the OS .iso
2. Open cmd/Powershell as admin
3. Run dism /online /enable-feature /featurename:NetFx3 /all /source:d:\sources\sxs (changing the /source argument to reflect the location of the install media)
Copy SIMSDocumentServerInstaller.exe from the old server (likely in .\sims\setups\ to the new server - the version you have will likely be old, ~~but as far as I can tell it's been the same version since at least 2009?! Version 1.580.12.0 is the newest I've seen - if you know different, let me know.~~ the newest I have seen is 1.580.99.1.
Open cmd/powershell as admin
cd to the location of the installer
Run .\SIMSDocumentServerInstaller.exe /INSTALL (this is case sensitive)
Click Next >
Select an appropriate install location - installing to the c:\ drive is fine as this is just the application not the actual storage location - then click Next >
Select a document store and a backup location - This should be on a separate DOCSTORAGE drive you've created (F:\, default allocation size, labelled DocStorage), or on the C:\ drive if preferred
1. The backup folder can live inside the docstorage directory
Communication Protocol: HTTP
Port: 8080
Click Next > and Next > again
Once installed, log in to Sims as an admin
Go to Tools > Setups > Document Management Server
Remove any existing Document servers and click Add
Enter a description and the FQDN of the Document Server
Test the connection
If you see a popup saying Server OK!, click OK then click Save
Navigate to the DOCSTORAGE location on the filesystem
Add a new folder, name it the hostname of the Document Server
Add a new folder inside that folder, name it the SQL Instance name (Sims if you followed the setup guide here exactly)
On the old server, navigate to the original DOCSTORAGE folder and go through the following directories: <hostname> > <SQL instance name>
Copy the SIMS Directory from the old server to the new server, placing it inside the <SQL instance name> folder
1. This might take a while - you can proceed with the SOLUS3 migration below if you wish, just try to not interrupt network or reboot prior to the data transfer completing
  Here's a robocopy command you can run to move the data a bit faster than explorer can manage it. Google the parameters used to make sure you're happy with them, and adjust source/destinations as required:
  robocopy "\\oldSimsServer.fqdn\e$\sims shared drive\docstorage\oldserver\oldinstance" "\\newserver\e$\sims shared drive\docstorage\newserver\newinstance" /mt:16 /MIR /copy:DT /ZB /FFT /R:10 /W:5 /dcopy:D
Test from the Sims client on the server

Sims.Net Clients

Getting the client to work on network devices is generally fairly easy, but sometimes you can hit errors when you try logging in from a client workstation. The most common error I've come across is SIMS Connection failed for Login <username>, reason: 0 which is incredibly vague but essentially means “I can't see the database!” - why, that depends, but typically it's some kind of network or connectivity issue.

Check the following on the server to increase your chances of clients working right away. Note that sometimes you won't need to do all of these and some may be already set up.

Log on to the new Sims server
Open SQL Server 2019 Configuration Manager
Select SQL Server Services on the left
Find the SQL Server Browser service
If the Start Mode is Disabled or the service is not running, right click it and go to Properties
Set the Startup type to Automatic
Start the service by right clicking SQL Server Browser and selecting Start
Expand SQL Server Network Configuration > Protocols for <InstanceName> on the left
Double-click on TCP/IP
In the Protocol tab, change Enabled to Yes
Apply & OK
Select SQL Server Services on the left
Restart the SQL Server (<InstanceName>) service by right clicking on it and selecting Restart
Expand SQL Server Network Configuration > Protocols for <InstanceName> on the left
Double-click on TCP/IP
In the IP Addresses tab, scroll down the bottom and make a note of the TCP Dynamic Ports value then close this window
Allow the port you've obtained from the SQL Server 2019 Configuration Manager step in the server firewall - don't forget to open any other required SQL ports (like 1433/tcp)

To point your clients at the new database location, you just need to update the connect.ini file the clients use. I opt to store the required configuration in the connect.ini file located in the Sims.Net install directory on the local machine instead of redirecting to a connect.ini file on a network location.

The important lines to update are the ServerName value, which should be the server name FQDN followed by the instance name (eg: SIMS-SERVER.internal.school.county.sch.uk\Sims), and if needed update the DatabaseName value to match the name of the database itself (eg: sims).

The official recommendation is to use SOLUS3 to deploy these files as well as client updates. However, as we all know, SOLUS3 sucks. So let's not bother using it (for clients). I instead set up a group policy object that writes both the connect.ini file and the sims.ini file to the applicable user devices. Just update the connect.ini file centrally, then force a gpupdate across your devices that have Sims.Net installed already. I also deploy these files as part of the PDQ Deploy deployment, which we'll cover later.

Hopefully Sims.Net is working for your users now. You can rest for a moment, weary traveller. The maintenance period can be over now. People can start using Sims again. Aside from a few things you should get to fairly quickly, outlined in the next section, SOLUS3 server will be needed, and you need to think about deploying updates, but these can wait at least a little while. Unless you have an update you need to roll out right away, of course.

Post-Install setups

Despite working now, some areas of Sims will eventually crop up with new errors, especially in secondaries.

Sims keeps some references to the server within the application itself rather than centrally on the database (why?!) which will need to be updated manually. You'll also note that as part of this migration, we haven't copied any of the files from the old Sims shared drive. There's a good reason for this - every Sims shared drive I've seen has either been completely unused, or full of stuff that, it turns out, isn't actually needed. I've only ever seen it mapped to all staff as a network drive write all staff having write access and as a result is used as a bit of a dumping ground.

There are also some directories that some of your users are probably used to using that may no longer exist in the new shared drive. This guide should get you 98% of the way to a fully functional Sims migration without expensive third party contractors... but it's these little obscure use cases that will catch you out! If you identify any additional areas where things are hardcoded or directories are needed, let me know and I'll get them added to this section.

Here are the ones I know about:

Sims Examination Organiser

Open Examination Organiser
Set the new folder structure if it complains it can't find the right folders (ideally use the shared Sims$ drive)
Go to Tools > School Setup > School Details and set the EDI directories to reflect the new setup if required

Cover

If you publish your cover timetable to HTML, you'll need to update the destination that these files save to if they used to save to the old Sims server.

You can do this through: Tools > Cover > Global Settings - section 3.

Sims Attendance Letters

If these are suddenly missing for your attendance team they will need to be located on the old server and copied across to the new one. Typically, these are found in:

S:\Sims\Attend

Running Sims on a workstation after an update and it keeps trying to install a bunch of stuff!

Odds are you have installer files in the location indicated by the SIMSSetupsDirectory= line in C:\Windows\Sims.ini. You don't need any of them. Delete them, move them, keep them elsewhere.

The only additional file you might want to install alongside SimsApplicationInstaller.exe is SimsManualInstaller.exe, which just extracts the manuals (the .pdf file user guides) to the workstation. But I haven't bothered with it!

SOLUS3 - unfortunately, it does make some stuff easier

Yes, we hate it, but it is convenient to have for getting update files and updating the server itself. At least, in order to do it more smoothly than a manual upgrade. If you're not lucky enough to have an application deployment tool like PDQ Deploy you will have a better time installing SOLUS3 in order to keep your clients up to date, too.

Whilst it is in theory possible to migrate SOLUS3 from an old server to a new one, I have only been able to successfully do this once. Each of the failed attempts failed for a different reason which often defied logic. So I just don't bother and install it fresh.

Install SOLUS3

Download the installer from dl.sims.co.uk (the password for 3.12.72, the version linked to, is currently YhgH7Y2TCvtcdG53 - they publish this password publicly (with the right Google-Fu) so I assume I can too?)
Extract all files
Run SOLUS3DeploymentServerDatabase.exe
1. Install the required prerequisites
2. Input the SQL Server and sa account details (SQL server = <hostname>.<FQDN>/sims) - click Test Connection to verify the details are accurate
3. The next window offers to set up a new SQL account just for Solus - this is a good idea, so generate a new account here. Record the details in your password manager.
4. Install
Run SOLUS3DeploymentService.exe
1. Install any prerequisites
2. Input the SQL Server and sa account details, NOT the solus3 user created previously (SQL server = <hostname>.<FQDN>/sims) - click Test Connection to verify the details are accurate
3. Leave deployment port as the default (52965)
4. If you extracted all files at the start of this SOLUS3 installation process, the Setup File Path for .NET Framework should be accurate. If not, hit Select... to change it to the location of the required installer file
5. The install location can be the default or change it to the storage drive if preferred
6. Install!
Run SOLUS3DeploymentServerUI.exe
1. Install any prerequisites
2. Input the SQL Server and sa account details, This CAN be the solus3 database user created previously but I have experienced some flakiness with this on occasion and now have reverted to just using the sa account here (SQL server = <hostname>.<FQDN>/sims) - click Test Connection to verify the details are accurate
3. Specify if you want any icons to be generated on the desktop/etc for the application
4. The install location can be the default or change it to the storage drive if preferred - ideally put it in the same place as the previous install if you're also installing it on the same machine
5. Install!

SOLUS3 Configuration

Run SOLUS 3 Deployment Server UI from the start menu or the desktop
1. Enter the Establishment Number and Postcode of the school, then click Verify
2. Once the school Address is populated, click Register
3. You'll be asked for a Registration Password - This would have been set on the previous install so you may need to get this from your current support provider or ESS directly if you don't have it
Once registered you'll be taken to the Settings > SOLUS3 page to complete the SOLUS config:
1. Service host: click the dots and move the server into the right column
2. Update repository (I create a directory, 'Solus3Setups' in the shared sims$ folder for this and share it as 'Solus3Setups$' - You will need to give 'SYSTEM' write access to the share) and click Validate
3. Click save
4. Next, go to Environment > Targets > Servers and click Add
  1. Server name: Find and select the new server
  2. Instance name: SQL instance name here (sims if you followed this guide)
  3. Binn folder: paste the location of the Sims Binn folder here (C:\Program Files\Microsoft SQL Server\MSSQL15.SIMS\MSSQL\Binn if you followed this guide)
  4. Click save and enter admin credentials to save the config
5. Do the same but for the Databases tab and the Services tab, adding the Sims File Server, pointing at the local location of the Sims shared directory (E:\Sims for example)
  1. You might want to do the DMS (DocumentManagementServer) too, in which case you should make a new Sims account with Returns Manager and System Manager permissions
6. Navigate to Settings > SOLUS3 > SOLUS and define any missing fields
7. Navigate to Upgrades > Update Library to see all your updates
8. Try downloading the version of Sims that matches the one running on clients
9. Navigate to Environment > Configure Workstations, then along the bottom select sims.ini
  1. In the shared directory, create a new folder (ini files for example) and then create a new sims.ini file.
  2. Copy the config from the old sims.ini file on the old server to this new file, checking that the data in it is still accurate
  3. Select this file in SOLUS, leaving the two installation options as use default
10. In the same Configure Workstations window, select the connect.ini tab along the bottom
  1. In the shared directory, create a new connect.ini file
  2. Populate this file with the relevant data - if you have working clients, just grab a copy from one of them
  3. Select this file in SOLUS, ensuring that you overwrite and that the other options are accurate. I like to set the ConnectionType to Trusted because by default, SQL authentication isn't as secure as Windows and this gives clients the option of logging in with their Windows session instead of relying on remembering/recording separate Sims credentials
    1. To set up an account to log in with the Windows credentials instead of needing to supply a username password combo, just change the username in System Manager to be <domain>\<username> (eg: internal.school\jsmith) and the interface will update
11. Turn on Solus by clicking the 'Off' toggle in the top right into the 'On' position. Add a couple test endpoints to begin deploying SOLUS agent and the current version of Sims

If at this point it works, you should be golden. If you need to deploy the SOLUS3 agent out to your workstations, do this now, however if you want to avoid the slowness of SOLUS3 and deploy the Sims.Net client via PDQ Deploy...

Deploy Sims.Net Client via PDQ Deploy

Say “No!” to SOLUS3 for client updates and use PDQ Deploy instead. I use this, and it cuts down our large-school-wide Sims upgrades from 3+ hours (with inevitable client issues) to being complete in less than 30 minutes.

I am using a licensed version of PDQ Deploy which makes this easier (multiple steps in a deployment! Retry queue! Yay!) but you could replicate it with the free version easily - just cut down on the steps to one job for deploying the file (which you do ahead of time) and a second to actually execute the powershell that runs the upgrade. You might need to get creative with how you do this on the free version. Once I can get at a license-free copy of PDQ I'll figure out the best path to achieving this.

There are some odd looking steps here - as we go I'll explain the reasoning for each.

Setting up the PDQ Package

Open PDQ Deploy
Create a new deployment and name it appropriately
Step 1: Copying SIMSApplicationSetup.exe to the target
Yeah, but why? Due to the way we need to execute this file to initiate the install, we copy it to the target first.
1. Pull the SIMSApplicationSetup.exe file from the extracted files from SOLUS and place it in the PDQ repository
2. Create a File Copy step in the Package
3. Single File selected
4. Source: Select the SIMSApplicationSetup.exe file in the PDQ Repository
5. Target Folder: Anywhere, I will use C:\Temp\
6. Overwrite Existing Files: Tick
7. Ignore Overwrite Errors: Untick
Step 2: Copy connect.ini
Yeah, but why? SOLUS3 deploys this usually, but we're not using that! You can also deploy the connect.ini file via Group Policy but this covers the “new device build” use case
1. Pull the connect.ini file from the server or one of your working clients and copy it into the the PDQ repository
2. Create a File Copy step in the Package
3. Single File selected
4. Source: Select the connect.ini file in the PDQ Repository
5. Target Folder: C:\Program Files (x86)\Sims\Sims .net\
6. Overwrite Existing Files: Ticked
7. Ignore Overwrite Errors: Ticked
Step 3: Copy SIMS.ini
Yeah, but why? Same reason as the previous step!
1. Pull the SIMS.ini file from the server or one of your working clients and copy it into the the PDQ repository
2. Create a File Copy step in the Package
3. Single File selected
4. Source: Select the SIMS.ini file in the PDQ Repository
5. Target Folder: C:\Windows\
6. Overwrite Existing Files: Ticked
7. Ignore Overwrite Errors: Ticked
Step 4: Kill pulsar.exe
Yeah, but why? If a target is running pulsar.exe - the main sims executable - sims will think it has updated but some files will be locked, so we kill it just before executing the upgrade. I see lots of people try and log in, see the Invalid Database error, and just leave it. If we don't kill pulsar.exe the upgrade will report as successful but will never truly succeed for that target. You may want to add additional .exe's here as you find them (please tell me about that) but for 99% of cases, shooting pulsar.exe in the head is enough.
1. Create a Command step in the package
2. Paste the following into the Details tab text box:
  tasklist | find /i "pulsar.exe" && taskkill /im pulsar.exe /F
  Yeah, but why? We don't just run taskkill /im puslar.exe /f on its own because if pulsar.exe isn't running when we try to taskkill it is considered a failure and will set errorlevel to 1, which PDQ will interpret as a fail. By executing the command in this way - passing piped output from tasklist - we never hit errorlevel 1 whether the task exists to be killed or not, because tasklist doesn't set an exit code, so PDQ is happy either way and will just continue on its merry way with the next step!
Step 5: execute SIMSApplicationSetup.exe via powershell
Yeah, but why? When I execute the install directly via the PDQ agent, about 15-20% of installs fail - no data gets written to the drive at all. I have no idea why this happens right now, but making powershell execute the same command results in 100% success rate (so far). It also has another advantage - typically PDQ will wait for a step to complete before moving on. If you launch the installer via powershell, the step according to PDQ is “run this powershell command” - once it's done that, PDQ will proceed to the next step, even though the installer has only just launched and probably hasn't started actually upgrading any files. The result of this small change is that you power through your PDQ Deploy queue way faster than if you had to wait for the installer to finish each time. Clients get their update faster and can get back to their jobs with way less friction and waiting around.
1. Create a Powershell step in the package
2. Paste the following into the Details tab text box onto a single line - note the ampersand (&) at the start:
  & "C:\Temp\SIMSApplicationSetup.exe" /S {QuietMode} [SIMSDirectory]="\\<server FQDN>\Sims$" [SIMSDotNetDirectory]="C:\Program Files (x86)\SIMS\Sims .net\"
  - Make sure you change <server FQDN> in the above command to the hostname and domain of the server (eg: sims-server.internal.school.county.sch.uk)
Step 6: Display install notification
Yeah, but why? This simply informs the user that the install is happening so they hopefully don't open the Sims application and break the upgrade. Staff may see Sims close (as part of a previous step) and try to re-open it, breaking the upgrade. You could display this message before executing the installer file rather than after if you prefer.
1. Create a Message step in the package
2. Type in a suitable message to display to users, something like:
  Sims.Net is currently being upgraded. Please don't click OK on this message, or open Sims.Net, until this message disappears.
3. Show for: 240 seconds
  Yeah, but why? The install will (should?) never take this long, but I'm accounting for edge cases and older hardware. A fresh install of Sims will take longer than an upgrade, as an upgrade only writes changed files and skips files that are the same.
4. Wait for user to click OK: Unticked

Deploy this to a client or two and verify that it works.

When it comes to rolling out a version upgrade, you just need to extract the files from SOLUS3 and overwrite the SIMSApplicationSetup.exe file in the PDQ repository, then push out the update to all your client devices. Easy.

Patches are a bit different - they are typically just self extracting executables which replace a few files in the Sims directory. These can be easily deployed with the File Copy package step in PDQ. Just throw together another package or add it as a step to the existing upgrade package as required.

Hope this helps speed your deployments up!

Implement But Never Move

2023-06-30

Implement But Never Move

Posted on Friday 30th of June 2023 at 00:00

I really appreciate well-written implementation guides for server-client software, but what really gets me excited is seeing migration guides for when you need to decommission that legacy OS and move onto something supported and current. Even the bad migration guides are great to have, but some that I've come across are works of art - numbered-list instructions that are clear and concise, incorrect assumptions busy sysadmins have during the migration are highlighted then corrected, screenshots where required, mid-migration tests to ensure things are going as smoothly as they seem... ahh, dreamy.

One of my personal pet peeves, however, is purchased server based software products that don't come with any migration guidance or instructions.

I'm spending a lot of time at the moment moving niche (read: there's only a couple of options and they all suck) and poorly made software from older server OS's to newer ones, and whilst some of them are fine and can be figured out right away, most of them have... issues. Typically you can just install $dumpsterFire fresh to the new server, dump the existing database and throw it over the fence to the new server (yes, some of this terrible software rEqUiReS tHe DaTaBaSe Is oN tHe SaMe SeRvEr), maybe I'll need to update some config files and point the new install at the existing database. One CNAME change later for the clients and you're golden. Simple stuff.

Sometimes, however, you need to perform some forbidden magical incantations, and the kick to the teeth in many cases is that nobody in $org will write down what they are publicly. So you gotta pick the phone up and get some overworked and underpaid $tech at $org to walk you through the process, shooting down the constant stream of bugs and errors that occur along the way due to the shoddy quality of $dumpsterFire with bullets of solidified experience (that'll no doubt be lost when $tech has had enough and leaves for greener pastures.)

Or, even worse than that... the $org requires payment for a migration because migrations aren't considered "support" and are "optional". Yes, it is support. And no, it isn't optional.

Dear organisations that explicitly hide their migration guides to force an already-paying customer to pay you yet again to migrate your horrible software from one server to another (immoral), that INSIST that you "must TeamViewer in to do this" (in business hours only!?), that there's no possible way anyone else can do it (false and stupid), that require DA/root accounts (you don't), that have to be installed on a Domain Controller (I'm crying), that MUST have unmitigated 24/7 access by installing un-licensed teamviewer (Oh no you won't)... There's even one software solution here which "requires" a physical server. In 2023. The software won't work in a virtual machine. (Oh, wait, spoiler: it works fine in a VM and has been working fine for over a decade)

To those organisations I say: No. Do better.

Because I'm writing this all down, I'm recording my screen when you connect to solve unhelpful errors in your hidden log files, I'm fixing your stupid permission requirements and immediately uninstalling any additional or third party crap that isn't a business requirement.

And I'm publishing those guides online for free.

Duct Tape

2023-06-19

Duct Tape

Posted on Monday 19th of June 2023 at 00:00

Back when I did end user support as my primary job I'd often ask myself "if there were no helpdesk tickets, what would I work on right now?"

The list I'd come up with was generally fairly short and changed each time I found myself pondering it, typically because the needs of the environment changed quite frequently, but it would also contain a few regulars - patch this, upgrade that, move that thing from there to there. Stuff that wouldn't have an immediately beneficial impact but should probably happen. These are all good things to do and I encourage them to be part of the daily workload of a team.

The dynamic entries in the list were always something to do with making something better, getting rid of an annoyance or frustration. These are the things that do actually have immediate ROI and I would argue they should have time spent on them even during high ticket load periods but often find themselves being ignored because... Well, it ain't broken. It's just not great.

The issue is, as hinted at, this list changes often. Sure you can keep notes, an ideas board or submit a ticket, but when you do finally get an hour to look at the list, what's on it doesn't seem that critical.

I just read an older post on rachelbythebay.com that summarised this and explains what it means in five words:

Look for the duct tape.

Read it (and whilst you're there, if you don't already consume that content take a look at some other posts - very valuable information and opinions contained within)

Essentially, find the things people have whipped up a quick workaround for and are using and fighting with now, whether that's within your team or another. Spend some time making that thing better, or resolve the problem at source if possible. You'll not only make that person or team happy, you'll also actively solve a now problem that will probably directly impact success, however that's measured.

Receiving Unsolicited Blogging Advice

2023-01-30

Receiving Unsolicited Blogging Advice

Posted on Monday 30th of January 2023 at 00:00

TL;DR: me post more!!1!

I recently read a blog post about a blog post and came to realise that I too struggle to post stuff on this very blog. I am conscious that I suffer from the curse of "I must make everything interesting!" but... maybe I don't have to suffer any more. Maybe I don't have to care. After all, if you don't like it, you don't have to read it.

I started this corner of the web to document my renovation and tech stuff for me and myself only. And Ben. (Hi Ben) But somewhere along the way it started to be about what other people find interesting, and... nothing really felt big or important or interesting enough. Hence the lack of posts recently.

So, a reset. Time to post more. They may not fit the predetermined categories, they may be completely uninteresting, but it turns out that not only is that okay, it's what I intended originally anyway. They're interesting to me, and besides I suspect I read these posts more than anyone else. Maybe, just maybe, someone else will find something a little bit interesting or useful too? That'll be a positive bonus.

The algorithms and view counts hold no sway. I forbid thee!

Interruptions In Tech Support

2023-01-30

Interruptions In Tech Support

Posted on Monday 30th of January 2023 at 00:00

I recently read some posts on the damaging effects of interruptions (in humans) and wanted to explore this in the context of my current job - a sysadmin across multiple k-12/ks1-5 educational environments - and offer some thoughts on how to change things.

First off, the core educational environment itself is pretty much built around interruptions. You generally have a single teacher in a room of a few dozen kids. I'm simplifying here, but the teachers essentially have particular targets to hit each lesson ("teach this thing", "make sure the group produces this result", etc) and these lessons are typically somewhere around an hour long, though from what I've seen these lessons are shrinking in length to squeeze more different material into a day/week. Teachers have to hop between different targets every hour (or less) as the group they teach changes (the students have to similarly hop, but across entire subject ranges. Is there any surprise that, combined with the usual suspects (advertising, media, social medias' infinite scroll, etc), young people have shorter attention spans?) Often these differing targets are in the same subject (Science, or English, or Mathematics, and so on) however different classes are at different points in the curriculum and even if some of those classes are at the same point in the curriculum (the students are of the same age) they're often grouped by capability - some classes need more time and effort and revision of material than others, whilst the top groups delve into subjects at a deeper level.

A teacher has an hour to achieve a goal before moving on to the next goal with a different group. During this hour, they will typically outline the thing to be done or learned, then work with the class to help them get there. This naturally results in questions (interruptions) continuously. It's in the teachers best interest to answer these questions well to ensure that individuals maintain pace with the group, but paradoxically the more questions that are asked the slower the group can move forward, as spending time explaining material again for the benefit of one means that those who do understand it already are not really learning for the time period it takes to get the one who asked caught up. Thus in my uneducated opinion, the priority for the teacher is to get the effectiveness of the first explanation of any material perfected (as closely as possible given the ability of the class) ensuring that most of the time most of the class (if not all) comprehends it to a suitable degree on the first attempt. This minimises interruptions and enhances learning for all, as the class learning velocity is kept high.

This problem is outside the scope of what I want to talk about but it is relevant, because one side effect of this we-must-acknowledge-interruptions behaviour is that I feel it becomes habit. Being interrupted a lot becomes the norm, and it is my opinion that this encourages the teaching individual to also be the one interrupting others more.

Let's go back to the classroom. You have one hour to teach a group of 33 twelve year olds about some algebra. You fire up your laptop and switch the projector on to find that no matter how you try you can't get an image to appear on the board. This unplanned interruption has immediately taken up brain power and, critically, time, even if you have backup plans for each and every lesson. As you have chosen to use the technology resource you have clearly decided that it is important, so we can assume that without that resource your teaching and the students learning will be less effective. You are after a fix for this problem quickly. The quality of your teaching is likely somewhat diminished and the longer this goes on the worse this gets.

So you do what anyone in your situation does - you try and get it fixed. This is where tech support comes in. You make a phone call or send an email to that tech you like, or if you're a hero, log a ticket on the ticketing system. The hero did the right thing - logging a ticket. No complaints from me there. (Pro tip for techs reading this: make every avenue of communication a ticket generating event!)

However emailing an individual or phoning interrupts the support techs. This is often warranted and is always understandable, and your job is a constant stream of interruptions so one more won't hurt, right? This is where, unfortunately, tech support suffer. We operate in the opposite universe. Your tech problem, as a teacher, is probably one of your biggest current problems. And we totally get that. Promise. We really do. But... it probably isn't our biggest problem and is almost certainly not something we're going to want to deal with right away.

An interruption to us does not progress anything, it in fact stops everything. If we are in the middle of resolving another problem, being forced to stop for a reason (whether it's to deal with a major issue or respond to a phone ringing or being pulled out of the moment by a name being called) causes us to disconnect from "the flow" and at best mentally change gears, at worse slow the brain CPU way the heck down. This is jarring and as evidenced by the many articles published about interruptions can in fact be damaging, both mentally and economically.

The more this happens the less effective overall an individual or a team can be. I argue that an ops team lead (Helpdesk Manager, IT Director, etc) should put work into minimising interruptions for the betterment of the team and yourself, regardless of your vocation or the objectives of the team. Here's what I propose you can do to help within your tech support team regardless of work environment... Though you will likely need managerial sign off on some or all of these. It's worth noting that I didn't come up with these, they're merely an amalgamation of things I've learned and tested which have worked for me. I am assuming you're part of a team instead of solo hero, too, though if you are solo then interruption reduction could potentially save you from burnout. Some of these might help.

Everything should (automatically) log a ticket

Make it easy to log tickets. Saying "no ticket no fix!" feels good but doesn't actually achieve anything for the organisation, except to piss off someone with an issue (and let's hope that person isn't a C-Level.) Let people call, let people email, let people visit, let people knock on the door. You can more than likely automatically create tickets in your ticketing system that come through to email, and if you can't, get a better ticketing system. Get a generic "techsupport@" inbox set up and configure your helpdesk to add everything sent to it as a ticket. Bonus points of you reply right away (automatically of course) telling the requester that their ticket has been logged. At least they know it's in the queue instead of sitting unread in some mystery inbox.

Ten CC's of Triage, STAT!

Yikes, you now have a flood of tickets. This looks bad. And hey, maybe it is, but at least now you know it's bad instead of it just feeling bad. Maybe management would be surprised to see that you've got 143% more tickets than you last reported, because everything is a ticket now. No more invisible work.

Anyway, I digress. Triaging tickets is essentially reviewing a ticket and deciding if it's a priority or not. There are many ways to judge this, but typically you want to take into account how urgently this needs to be fixed and how much of an impact it would have if it didn't get fixed.

A PCI Compliance audit deadline of two hours ago is pretty urgent - it's something that needs to have happened already. But... the banks aren't going to block all transactions immediately. Business doesn't stop because you're a few hours late on submitting a self eval form (this is not an endorsement to delay PCI compliance!)

A projector in a classroom has a pretty high impact - that's 30+ students (plus one pissed off teacher) per hour per day. That adds up quickly. And yes, it's pretty urgent too, but worst case the teacher can still teach like they did in 1985. Right?

Triage is important for two main reasons:

It helps you and your team decide on what to work on without having to decide on what to work on
Generally, you work on the thing with the smallest SLA time left, and if you have negative SLA counting down then you should probably get on that ASAP. Assign the Priority (the urgency and the impact) an SLA (say, high impact high urgency = 4 hour SLA, low impact low urgency = 14 day SLA - whatever you decide will be unique to you and the organisation and likely requires understanding managements expectations as well as your teams abilities)
Reporting, evidence, promotions, wage increase, redundancy protection, efficiency gains, areas of focus and more!
Having all this triage data to look back on will highlight where things are working and where they're not, but bigger picture style. You can use the data and the knowledge that you've gained to more closely align the team with the objectives *and reality* of the organisation. Perhaps when triaging your tickets you could also categorise them - hardware, software, licensing, accounts? Or go one level deeper - projector, laptop, desktop, TV, printer. Office, Web browser, MIS. After time passes you can really easily see where your team spends most of its time and where resources (like money) can be allocatied to reduce load in those areas.

Triaging tickets is generally not a full time job, so you could make this person...

Hero Assignment: Interrupt Shield

Designate an "interupt hero" - this can be one person or more and ideally they would triage tickets in the queue too. If you're lucky enough to be able to do this, put them in another room and leave everyone else in the relative peace and quiet of the main office. Make everyone else go to this smaller interrupt room for direct in person support.

They deal with all phone calls and in person visits and group emails. They also deal with any quick support calls (when not on a phonecall) that don't require them to leave their desk and that are interruptable, for example password changes, group membership updates, etc. You should rotate this person very regularly so you're not moving the "interrupt problem" to one person all the time, as that's a recipe for disaster. Let them do a day at a time, or two days if you use a hero team, rotating half the team out every day so handover and "current events" knowledge transfer (what were yesterdays big issues? What's on the radar today?) can occur more naturally.

Redirect everyones incoming calls to the interrupt hero team. This further reduces interruptions to the main body, though you may want to make exceptions for VIPs. Your interrupt hero team should be able to forward calls to you, but generally they will be able to deal with most quick things (and push to the focus team via a ticket if not, of course)

This has two advantages. First off, the toil is reduced for the team at large - your senior sysadmin isn't doing the third password change this week for little Joe - and secondly (and most relevantly) the rest of your team can focus for long periods of time on their work. Better work is done by most of the team. Better work is better results (whether those results are money/profit or a higher quality of education.)

Don't overload on active tickets

Don't assign anyone (including yourself) too many tickets. There is no perfect number - it's different depending on role, personality, type of work, and many more factors. But try and get a ballpark. I would suggest starting with no more than 10 non-pending tickets (for clarity, I define a pending ticket as one which is waiting for something outside the control of any member of the team) and only one should be worked on actively at a time. You can't install a printer whilst also diagnosing wireless issues. Pick one. Focus.

Low remaining (or negative/expired) SLA first

Plucking tickets from the queue because they look easy isn't productive, long term. But it's fine, short term. Don't sweat it, manager. But keep an eye on it.

Tickets should be worked based on their SLA, accounting for the available suitable resources (don't put the guy who sucks at DNS problems as the lead on critical DNS problems - train them? Yes. Rely on them? No.)

Sometimes it's nice to jump 70% down the queue and grab a few easy tickets. It feels like a break.

Just don't make it a habit.

Having repeat logs of the same call that has gone unanswered for three months is - you guessed it - yet another interruption. Old (the definition of which is, essentially, defined by your SLA - expired SLA? Ticket is old!) tickets should not exist (with very very few exceptions. Like budget availability.)

Don't judge people on their call closure rates

How many tickets I've closed vs the other techs means absolutely nothing. Get honest feedback from the source - ask the staff and, yes, the students, to rate the support. Then pay it attention. This feedback is so valuable. Whenever a ticket is closed, automatically send a followup email asking for feedback, and make it quick. You want the emotional immediate feedback, not the "dwell on it for three days and [calm down/forget]" feedback.

Was the support: [] Amazing! [] Ok! [] Horrible!
Optional comments here:

Negative feedback is the most valuable information you can possibly obtain

I argue that getting positive feedback means absolutely nothing, except as a "look how great we are, CEO!" line in the yearly review meeting. The feedback you really want, the feedback with actual damn value, is the negative feedback. And it's all well and good getting it. But make sure to pay attention to it, don't let it sit in the database doing nothing. Analyse it. Understand it. This is the closest you'll get to dipping your hand in the metaphorical technological currents of your organisation. You'll be able to feel the effect of the teams work, and when something goes against the current, you can fix it immediately.

This makes better, happier staff and students. Better happier staff and students equals better results. Which equals... you guessed it, fewer interruptions.

Interruptions aren't going away, but you can reduce them. And when you do, that firefighting feeling will decrease, work quality and rate will improve and as a direct result the stability of the environment should improve, too. And when the stability of the environment improves, the interruptions decrease.

At least, that's my experience.

There's plenty more about this on the internet at large, much better explained and with additional steps than what I have listed here, however I feel that these steps are the big hitters that might just give you enough time to step back and take a real close look at the bigger picture without worrying about where the next fire will start. And it'll take time. But it will work.

Don't Trust Copy And Paste Even With JavaScript Disabled

2022-03-22

Don't Trust Copy And Paste Even With JavaScript Disabled

Posted on Tuesday 22nd of March 2022 at 00:00

I've seen a few articles recently advising readers to not blindly copy/paste code from a website into your CLI directly because, with a small bit of JavaScript, you can overwrite the clipboard.

This is something that has been known about for a while and for some reason has seemingly resurfaced recently. The advice is to always paste into a text editor first to ensure what you think you have copied is actually what you have copied. However, I have seen comments about how you should disable JavaScript unless you need it in order to prevent this from occurring.

As awkward as this "disabling JavaScript" advice is on the modern web (and it does require some technical knowledge to enable just what you need) I agree, and in fact disable JavaScript by default. However, for this particular issue, this doesn't matter. You can achieve essentially the same thing without any JavaScript.

The stuff below isn't new. In fact, in the linked article is a link to a reddit thread where someone outlines this exact problem. But I feel that it can't hurt to reiterate. And explore!

So, for the obligatory warning: Don't paste anything on this page into a Powershell window. Don't paste it into anything but a text editor. The examples below shouldn't be harmful but… look, just don't risk it, okay?

Oh, and disable JavaScript if you want.

Malicious String - copy and paste the below example into a text editor (NOT a Powershell window)

echo 'hello,'

copy c:\inetpub\www\config.php c:\inetpub\www\config.php.txt -whatif

clear

echo 'hello world!'

Let's explore how we got here and what we can do about it.

The goal: hide text within an interesting string, so that when it is copied the user thinks they're copying one thing, but they have instead copied something else. Without JavaScript.

Invisible Text

This is easy with CSS - in these examples I am using inline styling so you can see what's being done. I've opted to use some example (probably-not-but-potentially-dangerous) Powershell, but anything works here. Bash, Python, SQL, .htaccess config, phone numbers, anything. The only limit is your imagination, and as always, what you can trick an end user into blindly pasting.

Malicious String - copy and paste the below example into a text editor (NOT a Powershell window)

echo 'hello,'; copy c:\inetpub\www\config.php c:\inetpub\www\config.php.txt -whatif;clear;echo 'hello world!'

Source: (You may need to scroll right to see it all)

<p>echo 'hello,<span style="font-size: 0;">'; copy c:\inetpub\www\config.php c:\inetpub\www\config.php.txt -whatif;clear;echo 'hello</span> world!'</p>

As you can see, setting the font-size to zero on the <span>'s with CSS simply makes the contents invisible in the browser, but highlighting it and copying it still grabs the 'invisible' contents. (We've not used display:none here as whilst that hides the text, it also prevents it from being copied.)

This could be the end of the article, but... we can go deeper. This example does require the user to blindly press return after pasting it to execute the command, which risks them spotting the potentially-malicious content before it's executed. In a config file this isn't really avoidable as far as I know (as the user will still have to save it blindly) but in the CLI...?

Can we use newlines to automatically execute the code instead of relying on a semi-colon (in this example) and the user pressing return? Let's have a go!

Invisible Text, with newlines!

Malicious String - copy and paste the below example into a text editor (NOT a Powershell window)

echo 'hello,'
copy c:\inetpub\www\config.php c:\inetpub\www\config.php.txt -whatif
clear
echo 'hello world!'

Source: (You may need to scroll right to see it all)

<p>echo 'hello,<span style="font-size: 0;">'<br>copy c:\inetpub\www\config.php c:\inetpub\www\config.php.txt -whatif<br>clear<br>echo 'hello</span> world!'</p>

We've simply added in some <br>'s that render as newlines. If you paste this into a text editor you can see it works, but the issue is obvious - the malicious string clearly has something wrong with it as it spreads over a few lines, which will probably raise an eyebrow or three. I purposefully haven't injected a newline after the last section, as whilst in our example the screen has been cleared, the end user still sees that they need to press return to execute, as they expect to. Psychologically I feel that this appears safer for them, and whilst they're probably wondering why their terminal history has vanished they might not think about it too much if they can see the code they copied seemingly hasn't even been executed yet.

The question is, can we include newlines, whilst not rendering newlines in the browser?

Spoiler: yes.

Invisible Text, Invisible Newlines!

The 1990's called and wanted to become relevant again. So I said, "okay, I'll give you one cool trick that'll have sysadmins peeved."

Yep, you guessed it, the classic web-dev nightmare - the <table> - comes to the rescue.

Malicious String - copy and paste the below example into a text editor (NOT a Powershell window)

echo 'hello,'

copy c:\inetpub\www\config.php c:\inetpub\www\config.php.txt -whatif

clear

echo 'hello world!'

Source: (You may need to scroll right to see it all)

<table style="border-spacing: 0;width:auto;">
  <tr>
    <td style="vertical-align: text-top;padding: 0;">
      echo 'hello,
      <span style="font-size: 0;">'<br></span>
    </td>
    <td style="vertical-align: text-top;padding: 0;">
      <span style="font-size: 0;">copy c:\inetpub\www\config.php c:\inetpub\www\config.php.txt -whatif<br></span>
    </td>
    <td style="vertical-align: text-top;padding: 0;">
      <span style="font-size: 0;">clear<br></span>
    </td>
    <td style="vertical-align: text-top;padding: 0;">
      <span style="font-size: 0;">echo 'hello</span> world!'
    </td>
  </tr>
</table>

This is the same example used at the top of this post.

So, some things are happening here. Let's step through them. Note that you may need to add, remove or modify these depending on what CSS rules the site in question is using.

border-spacing: 0; on the table - this removes all spacing from the borders of the table. If you don't declare this, your cells will take up space, so you'll end up with gaps between your letters which will look strange.
padding: 0; - Removes all padding from the cells. Similar to the border-spacing declaration above, if you don't declare this, your cells will take up space, so you'll end up with gaps between your letters.
vertical-align: text-top; - This will ensure the text in the cells always aligns to the top. The table (and <td>'s) grow to match the tallest <td>, and typically text will render in the middle of a <td>, so this just resets that.
width:auto; - we have declared this here as my WordPress theme makes tables 100% width by default, which ruins things in this case.

Because we're using a table and copying content from across multiple cells, you end up with an extra character at the start of every line except the first - the tab character. Handily, every terminal or console window I tried completely ignores this character, so we can too!

Remaining Challenge

The only issue I can find with this technique is not with the technique itself, but with the execution of the "secret" code - to anyone paying attention, it'll be obvious what's happened. What, your screen suddenly clear'd? That's suspicious. I've looked into a way to rebuild the CLI history on both bash and powershell and as far as I can tell it's not possible to do with any kind of guaranteed accuracy.

Any ideas? Leave a comment or send me an email, I'd be interested to see if the very execution of these commands can be hidden from the screen (.bash_history is another thing)

Light Up Spinning Top In The Dark

2021-12-31

Light Up Spinning Top In The Dark

Posted on Friday 31st of December 2021 at 00:00

Something about these toys appeals to me. The Kid got a few so I snapped some pictures, expecting a blurry mess. And that's what I got, but in a good way! I love how the projected red light looks in the longer exposure photos.

Tales From Tech Support 02 The Server Room Is Nuts

2021-08-11

Tales From Tech Support 02 The Server Room Is Nuts

Posted on Wednesday 11th of August 2021 at 00:00

I've said previously that things come in threes. This particular month in 2020 gave us three quite severe issues in the server room at work.

One - as the tide comes rolling in

We had some heavy rain in early October over a weekend. I spent the majority of that time in front of the fireplace until work on Monday morning.

Before I even got through the office I was alerted to a flood in our server room. I rushed down there to find an absolute mess. We are unfortunately strapped for space and also use the server room to store our spare equipment and some peripherals and consumables. We lost quite a lot of stuff due to water damage however most of it was old so doesn't hold much value. There were some switches that got soaked, and also my own personal Draytek I was going to use as a secondary VPN (ours wasn't great at the time) during lockdown in case of emergencies.

You can see in the following image a “tide line” around the walls of the room - we had about an inch of water in there that slowly soaked away. A phenomenal amount of water given the size of the room and the fact that there are sizeable gaps under the two doors that allow water to escape to neighbouring rooms

Somehow our servers didn't get too wet. Humidity was at 90%+ for quite a while though, and we have experienced several hard drive failures since which were likely related.

After three days of drying, ripping up carpet and sorting through our stuff we could finally get back to the normal slog. However we learned that the fitted Aircon unit in the room doesn't have a cut off for drying the air. It's either on or off, and only when manually set. We can't set a target humidity. In a server room you typically want to hover at about 50% - too high and the moisture in the air corrodes internals, too low and static electricity can more easily discharge in the dry air. Before ripping out the carpet and other detritus we'd hover at about 50-70% without any flood water in the room, but since then we've gone as low as 30%.

We need proper climate control and monitoring. And for the leak to be fixed… It has been an issue for a while which involves the design of the roof - water does drain away but if there's too much too quickly, or the drainage is blocked as was the case this time, the water overflows and somehow finds it's way to the ground through the second path of least resistance… Which just so happens to pass right through the server room about two feet in front of the cabinets.

Two - splashback

About two weeks after the first incident we had more heavy rain. More water in the server room. Luckily not as much water and because the room was empty and carpetless it dried out within a day. Unfortunately it came in through a slightly different part of the ceiling about half a foot closer to the server rack.

I'd love to get up on to the roof to try and figure out where this water is getting in. I'm told the fix is “new roof” but wonder if making a path for the water to escape by which doesn't pass through one of the most expensive rooms on the campus is feasible. Not a fix, certainly, but a workaround until we can move the servers (which we should be moving before the end of this year.)

Three - this is nuts

Finally (at least I hope) is this, which happened about a week after flood #2:

We saw him on the camera we set up after the first flood. We raced down there and eventually managed to coerce it out from its temporary home directly under the server rack before it could eat through anything. There have since been more squirrels find their way into the building but none have yet braved the server room. For fear of drowing, I suspect.

Although management doesn't particularly care about incident management and response reports, I find the whole process fascinating. So I wrote up an incident report for the squirrel invasion and it was quite entertaining to type out to say the least. So many puns.

Error B8bb2b3e On Hp Office Jet Pro 8710

2021-07-21

Error B8bb2b3e On Hp Office Jet Pro 8710

Posted on Wednesday 21st of July 2021 at 00:00

TL;DR: Try disabling "SNMP Status" for the port on the Windows print server in Print Management. No guarantee this will work everywhere but it appears to have worked for me. Give it a go, let me know!

Printer says… what, exactly?

Feel free to skip the rest of this post. It's just the sequence of events that took me to the supposed conclusion/solution.

I arrived at a customer site to find a stack of issues, but one in particular caused some confusion to my smooth-brained self. A HP Office Jet Pro 8710 printer, which worked up until last Wednesday, had a black screen with an error on it stating: There is a problem with the printer. Turn the printer off, then on.

I of course tried a reboot. The printer switched on, booted up and appeared to be fine. But after 5 to 10 seconds it would make a clicking noise, the screen would flash blue, then go black with the same error. I hate printers.

It looked like there was some white text on the blue screen so I recorded it with the phone in slow motion and picked up an error code: b8bb2b3e

Googling this took me to one relevant result, a post on the HP forum with no replies. Great. I hate printers (I may say this several times) and I hate an error code with no official documentation anywhere on the internet. So, what is one to do in such a situation?

Well, poke about, push buttons, and hope you figure out a pattern, obviously!

My immediate thought was something PrintNightmare related, however as the post on the HP forum was made in April I moved that away from the forefront of my mind for now and started looking elsewhere. As luck would have it, I noticed that the error occurred just as the Wireless light finished flashing (because yes, this printer is connected to a domain network wirelessly. Literally the definition of evil.)

I plugged in a network cable that I pinched from a nearby desktop and the printer stopped crashing. Great, progress! This building had a new wireless network installed a couple of months ago so I began poking at that but nothing had changed in the last couple of weeks. Instead, I moved across to the Print server to change the ports over to the new IP address the device has from the wired network assuming it was wireless related and… pop! Blue screen with b8bb2b3e again. My head flipped back around to PrintNightmare patch issues or something up with the server. I didn't really have a direction of travel from this point as Event Viewer had nothing of interest in it as far as I could see, but I decided to send the printer a Test Print just before it connected. I've seen before issues with printers where it'll process an existing queue of work and then crash out or error, indicating that the core technology (network, server, printer) is functioning but some feature is causing an issue, and that's exactly what I saw here. The device connected to the wired network, sent out the test print page, then immediately crashed out again. Historically I've fixed this (rare) occurrence by removing and re-adding the printer to the print server but I decided to poke around some more and try to nail down which setting was causing this (if, indeed, any at all.)

Luckily it didn't take too many attempts to switch off SNMP Status Enabled, reboot the printer, and not see any crashes. The printer is still working a few hours later (back on the wireless… grr. We'll run some cable into this particular office soon) and I have since checked Windows Updates installation date - the printer stopped working right after updates were installed on the server. The server was up to date prior to this month's (July 2021) patch release so it does look like something in the most recent set of updates caused this. And yes, I have ensured the server is not vulnerable to PrintNightmare by checking for those registry keys, so it's not that.

If I unveil the exact cause I will update this post.

Kaseya Says Yes

2021-07-14

Kaseya Says Yes

Posted on Wednesday 14th of July 2021 at 00:00

I just read the truesec analysis of the Kaseya VSA 0-day that hit the news earlier in the month. I love reading articles like this, but this one in particular I had to highlight.

The authentication... "bypass"... utilised as a first step: D'oh! How did something like that even get into production? The linked article has more details but essentially, if all authentication checks fail (when querying this particular file, not generally) instead of saying "Nope, you are not authenticated!" they instead say "Oh, you don't supply a password that we can verify? Ok, let's give you authenticated status anyway👍".

Logic failures like this generally don't happen in the ideal world because they're blindingly obvious, so allow me to speculate for the rest of this paragraph. I can only assume that a developer temporarily set this up to diagnose a bug or test a feature and simply forgot to flip it back to "fail by default". This is why peer review is important, though the rush to get things out to production works against this. It's too easy to miss this kind of thing in the modern world. It shouldn't be, but it is. There should be no blame here on any individual "- I suspect a process/procedure needs to be looked at, or a team needs to be better resourced.

Visitor Information Disclosure In Wp Statistics

2021-07-12

Visitor Information Disclosure In Wp Statistics

Posted on Monday 12th of July 2021 at 00:00

Just noticed this and when Googling it has been picked up already, so this isn't new, but the wp-statistics module (v13.0.8 for sure but likely other versions too) seems to be logging information into the “wp-statistics.log” file in the root directory of the site it is installed on. You can therefore access it and in some cases read the IP addresses of visitors to a site if they have the addon enabled by visiting domain.tld/wp-statistics.log.

You can block external access to it in the .htaccess file via:

<Files "wp-statistics.log">
Require all denied
</Files>

I've logged an issue on their github page, ~~hopefully they fix this soon~~ 2021-07-22: a fix will be pushed out this weekend according to the latest update on the issue.

A quick google dork will show up a fair number of affected sites, including some... potentially embarrassing ones.

Glpi Ripe For The Injecting

2021-04-23

Glpi Ripe For The Injecting

Posted on Friday 23rd of April 2021 at 00:00

I'll move on from GLPI eventually and start working on some interesting technical stuff, but today is not that day.

We ditched GLPI after we got hit by an accidental SQLi from HaveIBeenPwned - in short, version 9.4.5 is vulnerable to an SQL Injection flaw. You can exploit it by sending it an email (say, to helpdesk@company.tld) and once the email gets automatically turned into a ticket and assigned, the SQL will be executed. This affected us because the obscenely simple execution string was included in the header of the haveibeenpwned email notification.

I've just been poking around GLPI again (we have kept it around for non-end-user stuff, isolated and kept out of reach) and noticed that there was a “telemetry” scheduled task in the list of Automatic Actions which got me curious.

GLPI have decided to publish some of their telemetry data, which is nice of them. But it shows that there's still a significant number of users running 9.4.5 and older.

Of the installs that report telemetry in the last year (and only those installs on 9.2 and above do this), 14,313 are on a version at or below 9.4.5, whilst 26,985 are on 9.4.6 and above. Over 34% of GLPI installs are potentially* vulnerable to this painfully simple exploit but over 12% of installs absolutely are still vulnerable as they're on 9.4.5 exactly.

*Potentially, but I suspect only 9.4.5 is vulnerable - they fixed it by accident in 9.4.6 here which looks like a response to an issue that appeared in 9.4.5.

We learned a lesson with the GLPI issue - keep your software up to date. Though to be fair to us, it was up to date according to their website at the time. There was a newer version available (9.4.6) but that wasn't advertised anywhere.

I hope these out of date installs get updated. We know there's a lot of malicious activity out there, but at the same time... accidents can happen.

Iced

2021-02-08

Iced

Posted on Monday 8th of February 2021 at 00:00

I love how ice looks when it forms on something smooth and flat like glass. If I didn't have to scrape it all off in a hurry to get to work I'd stare at it for at least two minutes. What? It's freezing out there…

Tales From Tech Support 01 Percussive Maintenance

2020-07-25

Tales From Tech Support 01 Percussive Maintenance

Posted on Saturday 25th of July 2020 at 00:00

If you've worked in IT for more than a year you probably have some crazy tales to tell. I certainly do - with nearly 15 years in the field I have seen insanity and hilarity - more than I could ever remember. So I thought to myself, why not write it all down somewhere?

This first tale comes from back in my early days.

As a PFY, I was trusted with little more than basic desktop repairs and printer toner replacements - a fairly common slice of life for many IT bods. I was relatively fresh faced, a few scars but nothing major. Eager to learn and eager to please, I was often the first to raise my hand and take any challenging job, despite the vast gaps in my knowledge. Our team was small - three techs (two general helpdesk end user support PFYs, one mobile device repairs) with one very isolationist "Network Guy" (who would now be called a SysAdmin) and one "Database guy" (although their only qualification was knowing what "SQL" stood for.)

It came as quite a surprise when, early on a sunny Monday, we were told that the Network Guy was leaving. And he wasn't being replaced.

It quickly became apparent that the entirety of his duties were to fall on myself and PFY#2, who had been working at this place for a bit less than me. Networks Guy's last day rolled around without much issue, nor much communication from him - in fact at one point I asked for his help with a network issue and he told me "I don't care, I'm leaving!" So it came as a bit of a surprise (and equal amounts of relief) when he called me and PFY#2 up to his office to give us his handover document.

His handover document consisted of a single sheet of A4 with handwritten notes about a few things barely qualifying as useful. A few IPs and other miscellaneous details about servers and switches, the odd issue he knew about but hadn't fixed, and maybe a password or two.

We received the document at the door to his office (we weren't tolerable enough to go inside on this occasion?) and quickly shoo'd off to our regular helpdesk support duties.

That evening, he left and we never saw him again.

The following weeks and months were absolutely insane. I can't recall much about what happened during this time. Myself and PFY#2 managed to keep on top of most of the helpdesk support calls and make a start on untangling the network. We quickly found that most switches were essentially unboxed and plugged in without any config changes, servers were unpatched and had uptime into the hundreds of days (you could tell when the last powercut had been by looking at the uptime) and we were getting very close to the limit on resources - maxed CPU and/or RAM, HDDs filling up. Group policy was a mess, roaming profiles reaching into the tens of gigabytes with nothing preventing their growth… it was a mess.

To top it off, out the back of Network Guys office was another small closet containing almost all the servers we had. Neither of us had ever been back there, and when we did we found dust, cobwebs and equipment that seemed to be switched on but we had absolutely no idea what it did. Helpfully the single sheet of A4 told us some server names and serial numbers.

We were fueled by Red Bull and the long (long) days began to blur into one massive learning experience. To this day I have never learned so much so quickly as I did back then as we fought to keep the place running, the users happy(ish) and continue to learn as much as we could.

There was one event, though, that utterly stumped us.

Cheerily and awesomely smashing the helpdesk as we were, we suddenly had calls coming in about email being offline. After a quick check we realised that, yep, email was down. We couldn't RDP into the Exchange (2003) server either - something was wrong, clearly. Off to the dusty old Network Guys office we go.

We walk in, grab the single 4:3 CRT monitor in the room, stretch the cables across the room from one of the power extension cables with a spare socket to one of the nearby tables, and plug the crusty old VGA cable into the back of the absolute beast of an exchange server. I mean, this thing was huge. An old tower, black, solid steel everything. No idea why it was up in this first floor room when all the other servers were on the ground floor, but… whatever.

Flicking the monitor on we quickly saw everyones favourite screen. Yep, it's blue, and it signifies death. The BSOD.

We panicked a little, probably downed another Red Bull each, then got to work trying to bring this thing back online.

Remember: we were literally flying by the seat of our pants here, and had been four months at this point. We had no idea what we were doing.

First things first - switch it off and back on again. We held the power button down, heard a massive CLUNK, the fans span down and the screen went black. Take a breath. Switch it on again, wait for the BIOS, wait for Windows to start booting, wait some more…. BSOD.

Crap.

We try again, switch it off and back on. This time, we hear a horrible grinding noise as the machine spins up. We get to Windows trying to boot and everything freezes - not even a BSOD. Off it goes once more, switch it on, grinding noise, BIOS doesn't even finish loading.

Double crap.

Backups! Backups? There are backups, right? One of our jobs is to take the tapes out of the drive and swap them with the next numbered batch in the fire safe, surely we could restore this? But… Network Guy did the backups. Network Guy didn't elect to write anything down about the backups! We don't know anything about them! An oversight of the highest order!

We know the boss has the phone number of Network Guy, but we need to try fixing this ourselves first. We don't want him shouting down the phone at us like the one and only time we called him for help before this…

More Red Bull, more diagnosing. On the odd occasion we can get Windows to try booting, and sometimes we can get to the login screen using Safe Mode, but no matter how quick we are, we can never get logged in, and even this doesn't last forever. Eventually the server just stops trying to load Windows and we're presented with some error about not finding a HDD. The grinding at this point is still going on and we're faced with our fear that the grinding isn't a fan, but the single HDD that all of our email is stored on.

There's nothing else for it. We've gotta call up Network Guy and ask his advice. Neither of us want to do this though - he wasn't helpful to us when he worked here, and especially not when he was leaving.

Is there nothing we can do?

I remember the moment - we were stood either side of this huge hulking great server with no more options (that we are aware of at any rate). Our eyes meet, and without saying a word we both think the same thing at the same time.

PFY#2: "Shall we?"

Me: "I dunno… I mean it's not working so maybe?"

PFY#2: "I think we should."

Me: "Okay. Let's do it."

PFY#2: "Go on then, you can try first."

Me: "No way, you do it."

We look down at this ancient black server, lined with solid steel frame, sides and front, touched with the occasional bit of cheap plastic and the odd faded sticker.

I sigh.

PFY#2 raises his foot, and kicks the bastard right in the side, smack bang in the middle.

The grinding noise changes in pitch audibly. It's still there, still buzzing away in that audio range that you think you can put up with but slowly sends you insane without you realising it. PFY#2 reaches down and holds the power button in to switch it off. He switches it back on.

BIOS loads up, boots fully, screen goes black.

Windows loads. And loads. We stare. And it continues to load. The login window appears after what must have been at least 25 minutes. We're in shock. No BSOD. No lock up. Buzzing? Yeah that's still there, but the server has booted. What the f-?

We rush to a nearby office, get the first user to open up outlook. It connects. Emails from the upstream start flooding in to their mailbox.

We check our own, same thing - it's working.

We're buzzing. Our blood is filled with adrenaline, caffiene, sugar, and whatever the hell else they put in Red Bull, but it also filled with joy. We fixed a server by kicking it.

After spreading the word that email is back, we get right back to our helpdesk calls, of which dozens have appeared since our exchange issue surfaced.

Not long after this we do eventually employ a sysadmin who I work with to this day, but this exchange server didn't get replaced immediately. I can't remember exactly when it was retired, but it whirred on for a good year or more after this. We tried very hard to not touch it - it wasn't perfect, and we definitely had at least one more very confusing issue with it, which I'm sure I'll write about at some point, but the beast chugged on and kept our email flowing until it was eventually replaced by a younger sexier model.

Some say that if you can find your way into Network Guys old (and long repurposed) office, and if then you can manage to make your way into the back room, you can, on quiet days when email traffic is up, still hear the buzzing of that once-failed-but-then-recovered hard drive.

This is how I came to learn about and respect percussive maintenance.

Ditching Glpi

2020-07-23

Ditching Glpi

Posted on Thursday 23rd of July 2020 at 00:00

The recent bad experience with GLPI we had at work was the final nail in the coffin and, after patching the issue, I quickly began looking for alternative ticket management systems. We have wanted a new helpdesk for a long time - the support we provide has evolved over the years since GLPI was first introduced and with the covid-19 pandemic this support has seen yet another shift in the way in which we work. Not only are we doing things differently now, some unrecognised or unrealised issues have surfaced which we all wish to resolve or automate away.

We struggled through the worst of the lockdown but quickly identified some limitations with our existing way of working, namely that whilst our helpdesk did fine enough when it came to tracking tickets, it should do more for us. We spent a lot of time on the management of tickets and attempting to contact people just to get them to perform simple tasks. Why do we have to fight our helpdesk to achieve a goal, and why can't we have a system that would let us run these simple tasks (read: scripts) ourselves in the background, invisibly to the end user?

I did some reading and some thinking post-GLPI-exploit and realised that we are essentially an MSP. We provide support for departments within a school, each of which has different objectives, priorities, demands and tools. Plus, we support several primary schools on top of that, and they are their own beasts entirely with unique networks, hardware and software, let alone processes and requirements.

As we emerge from total lockdown to a lesser version, we are also going to need to do our normal jobs but much more efficiently. With a "remote first" approach to minimise risk to our end users (both staff and students, but also guardians and members of the public) I quickly decided to look into RMM tools instead of basic ticketing systems.

Given that we also had issues in other areas (namely monitoring and alerting, in that what we have is barebones and actually broke a month ago) I was looking out for a solution that would kill as many birds with as few stones as possible.

The requirements were that it had:

A ticketing system
Monitoring and alerting
Patching, installing and scripting capabilities
Easy remote support
A good price (hey, we're a school and don't pass the cost on to the end user, we don't have a lot of money!)

I found a list of RMM tools and their features on the /r/MSP subreddit and went through each one, checking videos, documentation and feature lists working out which would obviously not work, which might work, and which would absolutely work. I narrowed down the options to four potentials. In the end, only two of them had a "per technician" pricing model - the "per monitored device/agent" pricing model would end up costing us tens of thousands - so it came down to a war between the two. These were:

To be honest they were a close match. I preferred the look and feel of Atera although SyncroMSP was more feature rich. We didn't really need all the features SyncroMSP boasted though, and Atera was cheaper (we're on the cheapest plan) so ended up going with that.

Although we haven't rolled out the agent to every device yet (we're still mostly closed and until we have the agent installed have no way of deploying software out to machines not on our network) we have started using it heavily. So far, zero problems and we are all liking it. It has already enabled us to preempt some problems that would become tickets, solving them before they ever affect an end user or get reported. I'm looking forward to diving into it more, I am especially excited about the recently announced Chocolatey support, which seems to work wonderfully.

It's early days yet, hopefully we can become much more efficient and provide a better service. Only time will tell!

Haveibeenpwned Com Pwned Our Helpdesk Glpi 9 4 5 Sql Injection

2020-05-30

Haveibeenpwned Com Pwned Our Helpdesk Glpi 9 4 5 Sql Injection

Posted on Saturday 30th of May 2020 at 00:00

Last updated on Thursday 16th of April 2026 at 16:03

TL;DR:

I should say before we get started, the fault for this lies entirely with GLPI, I place no blame at the feet of haveibeenpwned.com or Troy Hunt for this issue. It's all good fun! Concerning? Oh, for sure. You can't help but laugh though. Obligatory XKCD.

On GLPI 9.4.5, creating a call (via the standard interface or email, etc) that contains the basic SQL injection string ';-- " will be logged normally with no abnormal behaviour, however if a Technician assigns themselves to that call via the quick "Assign to me" button, the SQL query will be executed. In the case of the example string given above, all existing calls, open or closed, will be updated to have their descriptions deleted and replaced with any text that appears before the aforementioned malicious string. You can of course modify this to perform other SQL queries.

This is fixed on 9.4.6, ~~however at the time of writing the GLPI Download page still links 9.4.5 as the latest update available, you need to go to github releases page to see 9.4.6.~~ [2020-06-03 update:] now available directly from the GLPI download page.

9.4.6 was released before I found this exploit, however the GLPI website still showed 9.4.5 as the latest version. As far as we were concerned, we were on the latest version. Credit goes to whoever submitted it first, however at the time I had no knowledge of this already being known and resolved. Here's a video showcasing the issue:

The Long Version - how I found it, or how haveibeenpwned pwned our helpdesk

GLPI

We use GLPI as our technical support ticketing system at work. There are better solutions out there and we're investigating others, however GLPI has served us well since 2009ish. It is a web based, self hosted PHP/MySQL application.

haveibeenpwned.com

A website that catalogues and monitors data dumps for email addresses. It collects leaked or stolen databases, analyses them, pulls out any email addresses and can be searched by anyone for free to see if your email address has been included in a breach. You can also subscribe to receive alerts if your email address, or an email address on a domain that you own, is included in any future breaches.

haveibeenpwned.com pwned GLPI

Around late April we upgraded from an ancient version to the latest of GLPI - 9.4.5. All was well until we received an email from haveibeenpwned to our helpdesk support address, which automatically got logged as a support ticket. This email alerted us to some compromised accounts on our domain which were included in the latest Wishbone data dump.

The above image is a screenshot showing the header of a standard HaveIBeenPwned alert email. This isn't an image rendered in the email alert, but actual text - characters make up the ';--have i been pwned? logo. This is important later.

I rushed to get the HIBP report generated to see who's data on our domain had been compromised by clicking a link in this email-turned-support-ticket. We got the report in a second email, which created a second ticket. I grabbed the data, deleted the second ticket (as we still had the original open) and perused the data. After doing the necessary work alerting any users to the breach of their data I went back to the original HIBP ticket, and realising I hadn't assigned it to myself did so and promptly solved it. All is well, time to move on?

Not quite. I and the other techs quickly noticed that every single ticket description had been deleted and replaced with partial header data from the HIBP email.

This immediately stunk of some kind of SQL Injection flaw and my mind raced as to what the cause was. I had a suspicion I knew... Unfortunately we were in the middle of business hours and due to Covid-19 are fully remote - we need a working helpdesk, and I don't have the priviledge of working on potential security issues in the day job. We restored from a backup taken on the previous evening (not too much data was lost, thankfully) and carried on with our day supporting our users.

Understanding the flaw

As soon as work ended, I grabbed an ubuntu .iso and built me a webserver VM. I had a feeling I knew what the cause of this SQLi was (check the header of the email shown above, you don't need long to figure out where the 'malicious' code is!) but wasn't sure how it got executed - the email was parsed correctly and tickets weren't affected when the email came in, it wasn't until around the time I deleted the second ticket and closed the first call that problems arose.

After building the VM with PHP and MySQL, I hopped onto the GLPI website and grabbed the latest version from their site, which is shown as 9.4.5.

After setting it up and adding some test calls, I forwarded our original HIBP email to a temporary account I linked this test GLPI install to. Once the email was pulled in I went through the same steps as I had done earlier in the day:

Generate the report (which I didn't do again via the link in the email, I just forwarded the original email to my test email account creating a second ticket in my install of GLPI)
Delete the second ticket
Assign the first ticket to myself
Close it

I checked my test tickets I loaded in there beforehand and lo and behold, they had all been wiped and replaced with the same content from the HIBP email!

I restored the VM to an earlier snapshot and went through the process again, pausing to check the other tickets at each step. I quickly discovered that the issue only occurs when you assign yourself to the ticket using the handy "Associate myself" button.

Making it malicious

The email data already wipes the content of all tickets, but as it stands it leaves a lot of junk data behind. I wanted to minimise the data required to exploit the flaw yet retain the same behaviour.

Another restore of GLPI followed, with more tests trying to determine the minimum amount of data needed to execute the flaw. I spent some time cutting down the email from HIBP and quickly found that the opening lines of the HIBP email were indeed the culprit - I managed to shrink the exploit down to six characters (';-- " - the space and double-quote at the end appear to be required though this could do with more testing) to achieve the same kind of malicious behaviour, in this case deleting all content of the descriptions for every ticket in the database. If you log the malicious call with this string as the title (or leave the title field blank - GLPI will then automatically add the contents of the description to the title, in this case the malicious string we have identified gets added as the title) the title on all other calls gets wiped too, however if you do include a non-malicious title in the malicious ticket the original titles on the other calls do not get modified.

Success! This is a pretty severe issue, and although it does require some user interaction you can easily hide this exploit in an innocent looking support call. GLPI supports HTML emails, which get rendered (almost) normally within the interface. Simply hiding the text in an attribute or the <head> or something will keep it invisible to the tech. You've just gotta wait for them to assign it to themselves.

In the end, this isn't a zero-click flaw but it is easily hidden. If you hide the exploit and it doesn't work out the first time (a tech doesn't assign it to themselves) you can easily try again with another ticket until it works. Odds are the techs aren't going to read through the raw HTML of each ticket looking for problems.

Reporting it - late to the party

I hopped over to GLPI's github page to check for an existing issue and log my own if one didn't exist when what do I see but 9.4.6! I check the changelog and find this:

Well, darn. I downloaded and installed the update and can confirm the issue no longer exists. Congrats to whoever spotted it first! Edit 2020-06-04: Twitter user @thetaphi took a look at this and found that it was spotted and/or accidentally fixed by a developer whilst fixing a separate issue.

As it is already solved I don't really want to dig through the code and find the offending line or develop the exploit further. Edit 2020-06-04: some people have taken a look after @troyhunt tweeted about this issue. It is interesting (concerning?) that something this simple got through to release, especially when you consider the way to initiate the exploit is by assigning yourself the call. Why does the call description get parsed at all here?

Either way - if you're running GLPI, make sure you're on the latest release. Or look for alternative software.

G Suite Migration For Outlook Error 0x8004106b

2020-05-22

G Suite Migration For Outlook Error 0x8004106b

Posted on Friday 22nd of May 2020 at 00:00

TL;DR: You're logging in using a non-primary alias for the account. Use the primary alias and your email will migrate smoothly.

We're migrating around 100 email accounts that were on Exchange over to Google. This has involved changing some peoples alias so that they match up with everyone elses (<initials>.<id>@<domain>) however we've added their old address as an alias so they can still receive emails sent to their old address (eg: <fullname>@<domain>).

Due to some political reasons we're unable to touch the exchange server, so we're using the G Suite Migration for Microsoft Outlook tool. It's been fine up until today where we started seeing the following:

Note the Emails - they are all failing! What was odd was that this was only happening for one of the techs. He must have been doing something wrong! We monitored their steps and it all looked fine though. Taking a look into the trace file we could see the error appearing as Failed with 0x8004106b. Googling this didn't result in any usable results, but as we were looking into it another tech appeared and just happen to glance at the screen as the original tech was attempting to migrate a user again. Our spreadsheet contained some info including the primary email address and the alias for us to check and the tech noticed that the guy with the sync issues on his users was using the non-primary alias to log in. After trying the primary alias, the emails migrated successfully.

Interestingly, the Contacts and Calendar data synced correctly.

Working With Covid19

2020-05-02

Working With Covid19

Posted on Saturday 2nd of May 2020 at 00:00

I don't appear to have had Covid-19, this post is more about working with it in the world, not working whilst ill!

Bit of a rambling post, I'm adding to it between jobs so forgive any abrupt topic shifts.

Schools are closed, but students still need to learn. As tech support for a school, enabling this has been... Well, easy to be honest. We're a Google school so the shift for our students (and those teachers that bothered to learn it...) has been relatively simple. But there's more to a school than enabling teachers to provide material and help to students. The challenges arose (and continue to become apparent) with all the other departments that enable teachers to do their jobs; wellbeing, admin, safeguarding, leadership, the list goes on.

For us in tech support, the whole team has been busy providing support for the schools in the local area. We're avoiding going on-site wherever possible, though this is not always avoidable. Remote support is not something we had prepared for, either. We've been using Chrome Remote Desktop to get on to other user devices who are also at home, but as nobody has local admin and Chrome Remote Desktop won't pass through UAC prompts we've been unable to fix a small subset of issues. Something to correct moving forward for sure - a remote assistance like product (something along the lines of GoToAssist or LogMeIn) would be swell, but it's a bit late to deploy it now.

We had to roll out laptops to our non-teaching staff, but unfortunately with limited devices available many staff simply don't have the ability to do much work that can't be done on a phone. Many people who said they have devices at home didn't account for the fact that their two-point-four kids and partner would also be needing it. We didn't take this in to account either and are paying for it now.

Some students also don't have devices, however the DfE is offering devices to those most vulnerable. We'll see how that goes, I can't imagine there's a stock of millions of laptops stored in a warehouse somewhere, but something is better than nothing!

I've wanted to work from home for a long time and have biased towards tools that can be used from anywhere. Rolling these out to a wider user base has certainly highlighted issues in our processes and changes we can make in the future, however for now we're... doing pretty well, actually.

The impact on education won't be felt for a long time, but there are already some interesting developments.

Some staff are arguing for the use of what are effectively spying tools to check up on both staff and students. Thankfully, this is being outright rejected by senior staff in our school. Others are demanding a lot of their teams, forgetting that some of them have kids and/or elderly and vulnerable relatives and neighbours to look after. This is having a direct impact on the teaching and learning - some staff are just exhausted. I'm getting emails from staff late at night and very early in the morning. Students are also submitting work at unhealthy hours.

I love working from home. Much more productive, fewer interruptions lead to better quality of work faster. It's great. But for some it's clear their home-life does not facilitate it well.

I'm hoping once the pandemic is over, we keep what we've learned and don't just rush right back in to how life was before. I'm not sure I can survive a full-time office job anymore...

Dead Wood

2020-04-30

Dead Wood

Posted on Thursday 30th of April 2020 at 00:00

Next to our house is large Oak (I think?) tree which has, over last three years, slowly died.

It's not on our property, but had grown over the house. We had some high winds earlier this year which resulted in a decent percentage of the small branches falling on to our roof and garden. No damage, but it was clear the tree was becoming more brittle and heading towards dangerous territory.

The owners recently organised to have it cut down. This was a sad moment, however I asked for a slice of wood from the tree. Our house sign rotted away a while ago and I had my eye on the wood from this tree. I'm hoping to use this slice to make a new sign at some point as a nice little tribute to the beast that has stood guard over our property for decades.

All I need to do now is figure out how to properly do this, or find someone to do it for me!

The neighbour will be using the wood in their log burner but has offered some to us, too. Doesn't look like we're going to be cold next winter, or perhaps the winter after depending on how long it needs to dry for.

One silver lining: it's now feasible for us to get solar panels!

A Leak Here And A Leak There

2019-12-10

A Leak Here And A Leak There

Posted on Tuesday 10th of December 2019 at 00:00

We're slowly eradicating our mould problems and it seems the universe has taken offense.

First off, our shower is leaking. The leaky pipes are within the wall and just happen to be directly above a power outlet located on the other side. The paint has bubbled up on this wall as the moisture slowly found its way out and down toward the source of electricity. We're no longer able to shower or we risk the wall getting more damaged and the potential for a fire to start. We noticed the paint bubbling, popped the side of the bath off (our shower is above the bath tub) and realised the concrete floor was soaked, the underside of the bath and the wood that helps support it were literally caked in mould. The water has also soaked into the concrete and found its way under the wall into the kitchen. This appears to have been happening for a while now as the kitchen units along that wall are also caked in mould across the back. We will likely need to replace the units in the kitchen, which isn't too much of an issue as we needed a new kitchen anyway, as well as dismantle half the bathroom to fix the source of the problem, which is also not too much of an issue as we also needed a new bathroom. Maybe £10k to do them both to a decent standard.

To add insult to injury, the cess pit decided to spring a leak. The container itself is, thankfully, fine. There's a broken inlet pipe somewhere, letting rainwater in. We don't have an issue with sewage leaking out but every time it rains the cess pit fills up pretty quickly. We could likely dig up the pipes and try to fix it but we need to replace the cess pit for a water treatment plant anyway. This is not a cheap thing to do - we're looking at a starting figure of about £8k, though this will likely increase due to some issues with the water run-off and our lack of land.

They say it comes in three's. They're wrong.

My car engine light came on. Hopefully this is just the sensor - clearing it hasn't worked so the next step is to replace the sensor itself. I'm not sure what the sensor is called but it checks the engine vibration and if it detects something amiss, alerts the driver. If the sensor is not faulty it's potentially a new engine, which with my old-ass car likely means a new car.

A day after the engine warning light came on, my windscreen cracked. Luckily my insurance covers the cost of repair minus £125, which I have to pay. Not too bad.

Though a few days after, my partner and the baby were in a car crash. Both are fine! However the car is likely a write-off. The other party admitted fault right away and their insurance will pay out the value of the car, but until that happens we're without a second car, which makes it difficult for me to fix mine. Not impossible, just difficult.

I guess it comes in six's? Maybe two sets of three.

I'm excited to get these problems sorted. Though they've all come at once, besides the written off car they're all things we planned on doing anyway. They're the last renovation things we need to do before we can start on the detailing - primarily, sorting out the network (allowing me to finally post something technical on here again)

The fact that these things are now active problems means we're more focused on fixing them. We could have paid to update our bathroom at any point but have been putting it off. Now, though, we're looking at getting it done sooner. The downside is that we don't have the money to do it all in one go so we do need to pick our battles in the right order (the cess pit first!)

Mould

2019-10-31

Mould

Posted on Thursday 31st of October 2019 at 00:00

We've had mould problems ever since moving in to this property. We found it under coving, under the old carpet, behind wallpaper and in the kitchen cupboards.

We've slowly been eradicating it - we had most of the issues resolved (except for in the kitchen, but more on that in the next post...) by the time we had finished decorating. However, it kept coming back in our bedroom and in the bathroom along the edges of the ceiling/walls. We had put this down to humidity. At night we breathe whilst sleeping, and in the bathroom... well, that one is obvious. I was getting frustrated with the constant mould recently and called up a specialist company.

They refused to take my money.

Instead, they gave me some (free) tips which, in hindsight, are quite obvious if I had thought about it. I appreciate that they did this instead of charging for someone to come out and do the work for me!

First off, they told me that a high humidity does not result in mould. Instead, where condensation forms is where mould forms. Condensation appears where a surface is cold enough for the moisture in the air to turn into water when it comes into contact with that surface. If you can prevent the moisture from ever getting into the air (essentially impossible in a home) OR warm up those surfaces, condensation won't form as easily, and therefore mould will have a harder time growing.

Heating a room does not achieve this on its own. As the mould was forming along the edges of where the ceiling/walls met (on externally facing walls, too!) the specialist company surmised that the insulation in the loft was not covering those areas. The heat was escaping through the plasterboard ceiling and not being held by insulation, cooling the area down. Water in the warm air would travel up, hit the ceiling barrier and condense there, dripping down the walls if enough formed.

Up to the loft I went to disover that this was indeed the case. In some areas (not coincidentally the same areas where we had the most issues with mould) up to two foot of ceiling was exposed. Luckily, there was a spare roll of insulation up there already. I used this to fill in the edges about a week ago and since then we've not had any more mould appear in those trouble spots. Early days of course, and we're still getting mould around the windows, but I have some thermal insulating paint to test there.

Ideally we'd replace our windows but can't really afford to do so at the moment - they're old and the seal has broken on some of them, which we will get repaired.

Sims Net Is Slow

2019-09-04

Sims Net Is Slow

Posted on Wednesday 4th of September 2019 at 00:00

I should preface this technical post by saying that I am in no way a database dude. I understand what they are and how they work at a basic level and sure, I've written some basic SQL queries in PHP or queried some stuff in MSSQL but as a typical “jack-of-all-trades” type I am no expert by any stretch of the imagination.

This is also relevant for any database, not just SIMS.Net.

What is SIMS?

Slow.

For those of you who don't know, SIMS(.Net) is a Capita application used by a whole bunch of schools to record data about their staff, students and their parents. It's a school MIS.

It's quite renown as being very slow. There are many posts on the edugeek.net forum complaining about this, and there are many suggested ways of fixing it. Okay, that's not entirely true. Lots of these fixes may grant you a slight boost, but the backend of SIMS was written a very long time ago (decades) and at the end of the day it's just a slow bit of software. That said, there are some things that can be done to help, and this post outlines one of them.

The Database File

SIMS is a Windows client-server application. The server side runs from a single database, though there are some additional features which have their own databases.

Typically, the schools IT department will not install the SIMS server. It's quite finicky and likes things set up in a very particular way, and is often handled by a third party organisation.

The client application uses a shared network drive for some data storage. Typically, at least in every school I have seen, this is set up on the same drive on the server as the database is stored in. This is generally bad practise when it comes to databases - you should put your database file on a disk/drive that has nothing else on it. No OS, no file storage, no nuthin'!

There are a couple of reasons for this. The most obvious is that if a file on that same disk is being written to or read from, the database isn't being used. Your application must wait its turn before the database can be queried. Schools don't really care about this as the slowdown is not noticeable by end users given the volume of non database data being read or written, but on systems where every mili- or microsecond counts this can be a big bottleneck depending on the frequency of use the other files on the disk see.

The second reason is related to the first, and it has to do with how the database file itself is initially set up. You can define the size of a database when you create it. You should really set it to be bigger than the expected end size of the database when it's time to migrate it to a new system or replace it. Typically, if you know that in 3 years you're going to migrate away from the system and it will have approx 10GB of data in the database, you will probably want to add 20-50% extra on top of this when you first set up the database to allow the data to grow into the file. So you go make a 15GB database and can sit happy knowing that you'll likely never touch the sides.

You can set the database up to dynamically grow once you've filled it up, too. This is called “Autogrowth”. You start with a, say, 20MB database, but this quickly fills up. MSSQL is configured to allow this to grow, so increases the file size of the database in chunks. These chunks can be consistently sized (add 1MB each time) or a percentage of the current size (eg: grow by 10% current filesize. 100MB database grows into a 110MB database, which then grows into a 122MB database, and so on)

The default growth size for a database file to grow by is 1MB. This means that if you have a heavily used database with loads of data going in often, MSSQL will need to constantly grow the database by 1MB each time. This is obviously going to add some overhead to operations, however there's another side effect to this when you have other files being used on the same drive as the database.

If you have a 50MB database, then write some files to the drive, then add more data to the database such that the database needs to increase in size, youre going to end up with 50MB of database data on the disk in a chunk, a bunch of files next to it, then next to that another chunk of database file data. You see a single 51MB file, but at a disk level there's one 50MB chunk, random file data, then another 1MB chunk. This is called fragmentation, and it means that for the hard drive to pull data out of the database it needs to move a little needle (if you're using old spinning disks, like us. What about SSDs? Glad you asked.) a greater distance and more often to get at the data you want. This slows things down.

What the Frag!?

Where I work, I recently found that the database had been set up with unlimited growth in 1MB increments. We went from a near-zero sized database at initialisation to what is currently a 26GB file. Each time we added data that pushed the database file to full, MSSQL would only ever add 1MB to the size of the file.

This, combined with the use of a network drive which has been configured on the same disk, has given us a horribly fragmented database!

There's a SysInternals tool called contig.exe which allows you to query an individual file or filesystem and show you how fragmented it is.

Following this guide I queried the database to see how fragmented it was. Keep in mind, here, that the author of that guide made a 3.5GB test database and purposefully fragemented it 2000 times to show a “heavily fragmented” example.

Here are the results from my test.

PS C:\temp\contig> .\Contig.exe -a "E:\MS SQL SERVER 2012\MSSQL11.SIMS2012\MSSQL\DATA\SIMS.mdf"

Contig v1.8 - Contig
Copyright (C) 2001-2016 Mark Russinovich
Sysinternals

E:\MS SQL SERVER 2012\MSSQL11.SIMS2012\MSSQL\DATA\SIMS.mdf is in 102951 fragments

Summary:
     Number of files processed:      1
     Number unsuccessfully procesed: 0
     Average fragmentation       : 102951 frags/file

Yeah. 102951 fragments for a single file. That's insane. Every time the database is queried it likely needs to navigate around the disks dozens of times to get all the relevant data, slowing things down considerably.

Fixing Fragmentation

We can use the contig.exe tool to fix this. It requires that the database is offline so I've had to wait until this weekend to do this.

I took the database offline (via the MSSQL Server Management Studio GUI) and attempted to defrag the database file.

PS E:\MS SQL SERVER 2012\MSSQL11.SIMS2012\MSSQL\DATA> C:\temp\contig\Contig.exe .\SIMS.mdf

Contig v1.8 - Contig
Copyright (C) 2001-2016 Mark Russinovich
Sysinternals


Summary:
     Number of files processed:      1
     Number of files defragmented:   1
     Number unsuccessfully procesed: 0
     Average fragmentation before: 102951 frags/file
     Average fragmentation after : 3 frags/file

After waiting anxiously for 40 minutes the results of the operation came through. We went from a database split into 102951 fragments to a database split into just 3.

I revived the database by bringing it back online, verified it was working, then carried on with my weekend. All in all, it only took me about an hour and a half on a Saturday to sort this out.

Results

Before I did the work over the weekend I took some rudimentary speed tests of the SIMS application to determine if the change actually had any effect.

I perfomed two tests three times. They wouldn't stand up in a court of law but they're good enough for my purposes.

I timed how long it would take to log in - hitting 'Enter' on the username/password window to getting the logged in GUI loaded - and I also ran a report on our on-role students containing their forename, surname, year & registration group code. A simple report.

Both of these tests were performed at approx 11am on a weekday. Nothing else was going on at the time (no large reports) but SIMS was actively being used by a few dozen staff. The results are in seconds.

Before defrag - Friday

Logging In

16.7S
7.1
9.9

Report

18.6
15.7
18.7

After defrag - Monday

Logging in

10.6
5.7
5.8

Report

14.1
11.9
14.3

As you can see, after the defrag has likely (assuming no unknown other reason for the slow results initially) shaved off a number of seconds from each run. The first login of the day is always the slowest as some stuff gets cached on your machine, but even that was sped up (by over 6 seconds!)

Overall, whilst not conclusive nor scientifically sound, I am happy with this approximate 30% speed boost and have begun looking into our other databases to see if they're suffering from the same fragmentation.

I would advise you take a look at your SIMS database, too. Hell, any database you have. If you're a database guru you likely know about this already and laugh at my realisation but it was genuinely surprising news to me, though obvious in hindsight. I know about file fragmentation, I just didn't think about it in the context of automatic database growth.

Being able to say “I've increased the speed of one of our most critical applications by between 24-40%” sure feels good. Though it should have never been set up this way in the first place…

Boiler High Heat

2019-07-31

Boiler High Heat

Posted on Wednesday 31st of July 2019 at 00:00

We had a bit of a warm spell recently. One side effect for us was that some of our sites boiler control systems started to panic in an unexpected way.

Each boiler is fitted with a fire alarm which links into the site-wide system. Unfortunately it got so hot in a couple of the storage-heater boiler rooms that the temperature sensor started flipping out thinking there were flames causing the heat. Luckily, we caught the alarm within the 3 minute silent alarm before emergency services were automatically contacted and the site-wide alarm sounded, prompting an evacuation.

Overriding a firealarm long term is not a good idea, so we had to find a way to reduce the temperature in these small, cramped, dusty rooms.

Our solution was simple, but I liked it a lot - turn off the heating element, then run around the building turning on all the hot water taps.

This refilled the hot water storage tanks with cold water. The taps being on carried all the heat away from the room and within minutes the room had cooled significantly. It is a bit of a waste of water, which is a shame, and there was no hot water in the building for the afternoon, but that beats evacuating out from our cool offices into the high-thirty-C outdoors.

The simple fixes are the best.

Witholding Number On External Calls On Avaya Ip Office R5 Manager

2019-07-29

Witholding Number On External Calls On Avaya Ip Office R5 Manager

Posted on Monday 29th of July 2019 at 00:00

It took some googling to figure out, but eventually I came across the answer to this problem across a combination of a couple of old forum posts. I'm writing it here on the off chance someone else needs this. Or if I need it again.

Log into the IP Office R5 Manager
Navigate to the Short Code tree
Find the ? entry - the feature name is Dial
Change the Telephone Number field to one of the following depending on what you want
- . - this will withold the number you're dialing from
- .S<number> (eg: .S01234567890) - this will show <number> when you dial

<number> should be your number. If I set it to a number I don't own, it just shows the one I'm calling from.

Sometimes a business has some system or piece of infrastructure that just works. This is a good thing! Unfortunately, some businesses don't like to invest in technology unless that technology breaks.

In related news, I recently had to figure out how to toggle a setting on some old software, Avaya IP Office R5. As someone who is totally not a phone/VOIP person this was a challenge. Add on to that the fact that the software isn't very well designed and you have me fumbling about trying to understand terminology and remember where obscure settings are just to do the basics.

This software was managing a 15 year old phone system, and since its inception had been showing as a withheld/private number when calling out to external phones. Not sure why this was ever set, but it's off now.