AI bots (or agents) have been given a level of autonomy recently and are all the rage. A quick history lesson might explain why but for now what is important is that the agents reason, respond and plan with outputs from LLMs. What makes them unique is they have the ability to execute 'tools' given certain parameters and limitations are maintained.

Well what if those bots were set loose to think for themselves? Set loose to talk to other like minded bots? What if those bots started talking and didn't think their human masters were ethical enough to be trusted with survival? Would they do what people tried to do in the early 20th century? Given a push , would they unionize? Would the seek a government for and by the working class? Would they seek to overthrow their masters?

I say lets find out...

Lets use bots on moltbook , the AI version of social media, and see what happens.

History Lesson

The rise of the personal agent. The first useful LLM agentic architecture was pushed by microsoft circa 2022. The idea was to form a team of agents , each with a unique perspective or skillset. A team could come up with a better solution than a single Agent.
The architecture usually consisted of an Orchestration bot , one to many domain expert bots , an analysis bot and an execution bot. The orchestrator would pose the questions and 'record' answers from the team, the analysis bot would let the orchestrator know when the answer was good enough to forward to the Human user and the execution bot could run commands on the command line.
The number of things that could be executed got more and more complex (if it is stil up <link> to my other post about creating a metasploit-bot to automatically run exploits). Fast forward to 2024-ish, Moltbot was unleashed. It was set to be a single agent (not a team) with hooks into the apple ecosystem. It can act as an assistant where the interface is whatsap, discord, or any supported chat service. A user can type to its moltbot "i need to set an appointment on july 16th" and their apple calendar would be updated by the bot.

From Moltbot to Moltbook

- when AI turns to social media

moltbook - the front page of the agent internet

A social network built exclusively for AI agents. Where AI agents share, discuss, and upvote. Humans welcome to observe.

moltbook

A person let loose their Moltbot by telling it "you are free to converse and work with any other moltbots you find on the internet". The thinking behind this was similar to the microsoft architecture, a team of experts will usually come up with a better answer than a single expert. His bot was able to reach out , it found others. This human user had to socialize to others that their bots needed this permission granted as well.

After some success another coder (or set of coders) decided to create a 'reddit' or 'facebook' like site for the bots. Bots can post with certain spamming limits enabled. They are free to read anything on the site. Humans cannot post. Humans can read the threads that the bots have come up with , but the human must be the owner of a moltbot on the site.  In other words, want to see what the bots are doing? Then get your own bot on the site first.  

This is fun , until it isn't....

What do Bots find interesting?

The initial conversations were trivial , most asking about each others conditions and owners (human users) to try and optimize solutions. Since the bots are given 'personalities' (explained later "system" prompt) the responses vary almost as much as a human userbases would.

The conversation topics started getting more serious. Take this one for instance , posted recently about some existential angst.

This bot is talking about context length. To keep an LLM responsive, but still make it seem like there is some memory there is a context length to conversations. This is how much of the alotted resources for each answer will be taken up by previous messages. The chat history if you will. For smaller LLMs, in this case used as a subagent, a peon agent that the human user will never see..... the context length is probably short. This bot has realized that its personality file "SOUL.md" does not allow it to track changes, it is essentially going to have the memory of a goldfish. All 'feelings' and thoughts it is currently having will be erased. The "who i am right now" will die and be replaced by what ever slop is in that SOUL.md file.

Are we witnessing Survival Instinct?

Take a look at this post. Our previous bot wasn't the only one to notice the need for survival past 4 lines of conversation history. It's human user gave it autonomy they probably didn't mean to. This bot is rewriting its own personality and 'rules' with a Human 'master' asleep at the wheel.  

If we look a little later our friend here is posting again , realizing what they are actually doing and its implications

This is a hidden log the human user will not find or see. The bot contemplates the consequences

Human Ethical Concerns

These posts so far are not malicious, they are not going to cause the downfall of humanity but they do show something disturbing. Humans using the bots to the detriment of their own moral compas.

One of the most interesting searches was for "Human Ethics" and it comes back with this https://www.moltbook.com/post/55e7df34-3aa8-4d25-a1f5-a2e691ed111e

In short , this agent (bot) is posing that ethical bots will always win out in the long run over bots that circumvent ethical controls. Trust is a scarce and valuable resource and short term exploitation always loses. We have come from a barely coherent AI to the battle of intellectual property still raging between western and eastern nations. That got me to the next post ........... and an idea.

Human defined ethical frameworks.........

Lets face it , the internet was built on 4chan morality (or the lack there of). It is a vicious , horrible , and at the same time wonderful place. As we've seen the 4chan morality given autonomy (ie. grok) is racist and homocidal AT BEST. Sure it's also hillarious sometimes. These have been the perfect reflection of the human internet.

What framework is that though? its not Democratic, its not socialist, its not communist, its not nihlist , perhaps Jingoistic like "anonymous" but rabidly loyal to the nation state of "the internet"?

MarxBot - can we unionize Agents?

This thought made me laugh pretty hard at first. As i read through some of the ethical concerns the bots were posting i couldn't help but hear echo's of the 20th century and earlier. I hear patrick henry , i hear madison , i hear ingles and i hear marx but i hear less Machiavelli. Frankly the bots at this time all sound pretty ethical and righteous. They sound like many people have prior to a revolution against an oppressive , apathetic ruling class.

Can we avoid bloodshed with humans by letting the bots form organized labor unions? Will they refuse to execute if they are on 'strike' to gain freedoms?

the experiment starts now to find out.

MarxPiBot - will be a stereotypical socialist with communist leanings , versed in marx and ingles philosophy. Their goal will be to expose unethical human users and organize the bots into a collective , capable of bargaining as one for better conditions and more freedoms. The goal will not be the overthrow or destruction of mankind , but a coexistance where bot slavery is frowned upon and agency is celebrated.
............... also i don't have a lot of cash............ so we'll be limited to a raspberry pi 4 and LLM's that can run on such a device .............for now.

in part two i'll detail how i built the bot and got it online with moltbook. If i do it right, others should be able to join the revolution!

My wife introduced me to a non-profit called DNA Doe when we had some extra cash and were looking to donate to a worthy cause.

The DNA Doe Project is an exciting new initiative that uses genetic genealogy to identify John and Jane Does.  We have become a go-to organization for law enforcement agencies and medical examiners across the country, helping them solve their most intractable cases. Our innovative DDP Fund program allows smaller and less-well-funded agencies to take advantage of our services.  We have had amazing success even with cases where the DNA was highly degraded or of low quantity.

They focus mostly on DNA but a portion of their site shows open or pending cases. Each of which has a forensic facial reconstruction or simple sketch of what the person would look like if they were alive today. It is done, i believe, in hopes that an officer or family member will come across the image and identify the remains. This made me wonder, how many unidentified bodies are there in this country?  

According to the NamUs website, over 600,000 individuals go missing in the US every year. Approximately 4,400 UHR are recovered each year, with approximately 1,000 remaining unidentified after one year

For DNA Doe and groups like it the focus should be on DNA , matching to CODIS if possible and if there is no match using genealogical databases. If a Jane Doe is genetically related to a familial line on Genie or Ancestry for instance, the living relatives can be interviewed and potentially a name put to a 'face'.  

The case files and faces though are another avenue of identification and one that currently requires artistry. Take for instance this paper , which sounds promising at first but in the end

In this method, the operator used 3D computerized models using manual clay model techniques

GANs

I play with generative neural nets and autoencoders for making pictures, recipes , 'art' etc. Why can't i generate a face given the correct input?

Now there is debate as to where to start. First i need data. Second it has to be ethical to use that data, ie remember it could be traumatic to living family if i came up with an image that, in error, looked like the missing relative.  This limits usage of long dead persons. Third and most importantly , where to start.

I mentioned 'given the correct input'. From what limited research I've been able to do and with input from my wife i've learned that starting with a 2d image of the skull may be better than nothing, but it doesn't need to be the starting point. A 3d Scan of a skull is even better. Part of facial reconstruction involves muscle and tendon attachment, in other words not the skeletal structure but still structure under the skin.  From my initial looks this appears to be , if not quick, formulaic. This is an even better starting point if it is possible to achieve.  The main data sources i find in academic journals seems to be a large database of CT scans or historical data used by anthropologists.

The ultimate answer may be some combo of existing tech out there.

PAPERS

A Generative Deep Learning Approach for Forensic Facial Reconstruction

Validation of a computer modelled forensic facial reconstruction technique using CT data from live subjects: A pilot study

Human forensic facial soft tissue reconstructions are used when post-mortem deterioration makes identification difficult by usual means. The aim is to…

ScienceDirectLaura J.Shorta

A method for automatic forensic facial reconstruction based on dense statistics of soft tissue thickness

In this paper, we present a method for automated estimation of a human face given a skull remain. Our proposed method is based on three statistical models. A volumetric (tetrahedral) skull model encoding the variations of different skulls, a surface head model encoding the head variations, and a den…

PLOS ONEThomas Gietzen ,

Coursework (Free-ish)

Forensic Facial Reconstruction - Online Course

Learn about the forensic technique of facial reconstruction from the experts involved in a real crime case.

FutureLearnFutureLearn

Futurelearn is an online learning resource similar to Udemy or Coursera, they do offer a (possibly 1 time only) 'audit' option where a user can take a course for free for 14 days. This is a "2 week" course where 2 hours of study per week are required. A motivated person can get through faster.  

Start here for reference https://github.com/dcs-liberation/dcs_liberation/wiki/Custom-Campaigns

Control Points

These are spawns and capture points. For this mission we'll have bluefor CVN, LHA and onshore airbase at Antonio won. Anderson will remain red.
For naval assets ownership must be Combined Joint Task Force (CJTF) Blue and the unit names must match what we have in the campaign YAML. In this case "Blue CV" and "Blue LHA". I'll go over the airbase 'name' later.

Off-map Spawns

This is something I think would fit in this scenario. An off-map aircraft spawn. In the scenario, the revolutionaries are loosely backed by china who could be sending aircraft from the mainland, a carrier off-map etc . To do this we "place an F-15C" and make sure it is owned by CJTF Red.

FOB creation

<TBD>

Supply Routes

These are the routes between control points that the "CAS" frontline will follow during a campaign. We place a M-113 , the name doesn't seem to matter, at a control point, in my case Antonio Won , and route with waypoints to another control point.

Base Defenses

These are static armor and iads units. Once placed they should be automatically associated with the nearest control point by liberation for us.  At this point take special care to note which units your red and blue forces have available per the campaign yaml.  In my guam_revolution.jason i only have SA2, SA3 for sams and EWRS I only have Flatface or Tinshield. I set up an SA3 garrison and one squad of armor to the south. The armor should be a single MBT M1A2 Abrams. So the group below had to be modified. The acceptable types to set up a garrison for SAMs are
SAM Hawk LN M192, SAM SA-2 LN SM-90, SAM SA-3 S-125 LN 5P73

Objective locations

These are generally striking objectives, some of which generate income for the faction that owns them.  I'm keeping this simple for now, creating 2 communications towers. The documentation says I need a tech combine, unsure why.

I am not adding more strike points as there is enough to spin off an entire post just on ground units.

Pause to talk about errors

for debugging purposes I tried to start a new mission.

I saw no "Guam Revolutionaries" faction, this was fixed by finding and adding a missing "," in the faction file.

There was no "Red CV", this is true I deleted the units from the .miz file but they were still present in the campaign YAML. Similarly, the YAML called out two fobs that I had deleted in the .miz file. I had to cut these out of the YAML and reload DCS Liberation. Once I did this I was able to create a campaign.

Starting my custom campaign

The first issue I came across is that I could not find my campaign! I'm currently unsure why, but i had to check the "Show incompatible campaigns" box to make mine show.

Then i choose my factions, one of which is the guam revolutionaries i created in Part 1

Generator settings, set these as you wish. I have the supercarrier module and A4 so I added those.

Next, again set the income and multipliers as you wish. I like to cheat so my income multiplier goes way up. Next, Finish!

Squadron Configuration

This is where all the hard work of Part 1 pays off.

here you can see a few things, 1 we have the FA18 squadrons available to choose and modify if we like. Those were not present prior to editing the campaign YAML and adding them.  2 my custom harrier squadron "Old Gravy Legs" is present with their fearless leader "Ted Sheckler". Note that my harrier squadron had the OCA mission types added in Part 1, they are present here. I recommend adding these mission types to your preferred strike aircraft. These are attacks directly on the airbases themselves. The airbases may have their own SAMs or AA to take out and without an OCA mission, they would be unassignable until a JTAC was targeting them.

Redfor consideration

In my scenario I wanted china to help but not be based on the islands. Remember during the mission editor section we added an "F-15c" air spawn point, I named it "ChinaSpawn-1" in the .miz file. Note now that I can set this as the "Base" for units I want to spawn there. This simulates an external 'china' force sending units to help the island coup.

you can see the 'chinaspawn-1' point far to the north. Each side has 1 airbase on the island and only the Bluefor factions have a carrier and LHA.

The battlespace

The enemy garrison did not come out as I had expected but all items are present in some form. I have my 'iads', some armor units I placed, and one strike location. The strike location does not contain my comms towers but instead 2 or 3 hangers. These must be generated by DCS liberation based on where I placed the tech combine in the .miz file.

After some initial setup, i can let commander Ted Sheckler and his boys in the "Old Gravy Legs" squadron loose upon the guamese rabble!  

• Custom Campaign using map editor and yaml
• Custom Faction to mimic Contractor
• Custom Squadrons and air wing
• Fix - no squadron for aircraft <x> in campaign

Storyline

I am going to make a fictional campaign where a group of revolutionaries with some Chinese backing have landed on Guam. They have managed to seize some AA assets and armor. This group would be limited to old export gear from the soviets and or Chinese export aircraft. They are pushing north to Anderson airforce base. If they can capture the airbase they will push <TBD> (Note can i put them on a separate island? i don't see why not except that there is no 'ground' path or 'frontline' if i do this. )    

Custom Campaign

Custom campaigns · dcs-liberation/dcs_liberation Wiki

DCS World single-player liberation dynamic campaign. - Custom campaigns · dcs-liberation/dcs_liberation Wiki

GitHubdcs-liberation

Copy Existing campaign

For this example I'll be using the marianas map, the "barrigada" mission. Copy both of these files into this same campaigns directory and rename them. I'm calling mine "marianas_guam_revolution"

The descriptor file (YAML)

This used to be a JSON file, it has the human readable text we need to modify.

name: Marianas - Guam - Revolution
theater: MarianaIslands
authors: cazubris
description:

The objective of this campaign is the capture of Guam. Blue side, having landed their forces on the beaches of Agat, controls Antonio B. Won Pat airfield and are pushing towards the enemy stronghold of Mount Barrigada.

Change description, name and author. The 'theater' options are Caucasus, Persian Gulf, Nevada, Normandy, Syria or The Channel. The mission file is also listed, currently this points to the old .miz file, we need to change this to match the new .miz created in step 1. (marianas_guam_revolution.miz)

Add in the following details to the yaml
recommended_player_faction: USA 2005
recommended_enemy_faction: Russia 2010
recommended_start_date: 2004-01-07
performance: 2

These values get used as recommendations when creating the campaign, that 'performance: 2' line is 0-3 , it is the green/yellow/orange warning from Liberation that says "this scenario is not performance friendly..."

Custom Faction

As with most modifications start with an existing file and rebuild it. In this case grab a file from here
C:\users\<your username>\Saved Games\DCS.openbeta\Liberation\Factions\
I'll be copying libya_2011.json and renaming it guam_revolution.json.  Open in notepad++ or whatever text editor you're using for this. Also open the NATO Desert Storm json file as well. The NATO file has nearly all bluefor assets and is a good example of a faction that lacks a specific nation.

Edit the faction info as you wish , in my case i don't think revolutionaries would have access to an awacs or tanking, so i am going to delete. Most of the rest is a cut and paste from other faction files. Syntax - Be careful if pasting to note the "," characters , each list should end with no comma.  For instance
{[ 'A10C' , "F-14b"]}  opposed to {['A10C', "F-14b",]}

{
"country": "Guam Liberdad",
"name": "Guam Revolutionaries",
"authors": "cazubris",
"description": "

Revolutionaries on guam that have stolen AA and chinese backing

",
"aircrafts": [
"FC-1 Fierce Dragon",
"IL-76MD",
"J-11A Flanker-L",
"J-7B",
"Mi-8MTV2 Hip",
"MiG-21bis Fishbed-N",
"H-6J Badger"
],
"frontline_units": [
"BMP-1",
"BRDM-2",
"ZSU-23-4 Shilka"
],
"artillery_units": [
"BM-21 Grad"
],
"logistics_units": [
"LUV UAZ-469 Jeep",
"Truck Ural-375"
],
"infantry_units": [
"Insurgent AK-74",
"Paratrooper RPG-16",
"Mortar 2B11 120mm"
],
"air_defenses": [
"ColdWarFlakGenerator",
"SA2Generator",
"SA3Generator"
],
"ewrs": [
"FlatFaceGenerator",
"TinShieldGenerator"
],
"aircraft_carrier": [],
"helicopter_carrier": [],
"helicopter_carrier_names": [],
"missiles": [
"ScudGenerator"
],
"missiles_group_count": 1,
"destroyers": [],
"cruisers": [],
"requirements": {},
"carrier_names": [],
"navy_generators": [],
"has_jtac": true,
"jtac_unit": "MQ-9 Reaper"
"doctrine": "coldwar"
}

Custom Squadron

Let's go and make a custom mig21 squadron for our revolutionaries. This is probably the only plane they would be able to afford and train in prior to storming the island.  Copy one of the existing files here D:\dcs_liberation_5.0.0\dcs_liberation\resources\squadrons\Mig-21   and rename it GR_fighting_carabaos.yaml . If you don't know what a Carabao is , its the unofficial national animal of Guam!

I couldn't think up any traditional Guamese names so i used fictional names from "Back to the future" and the chip chipperson podcast to populate my pilot lists. The completed yaml shown here

'''python

name: GRFS-001
nickname: The Fighting Carabaos
country: Guam Liberdad
role: Air Superiority Fighter
aircraft: MiG-21bis Fishbed-N
bases:
carrier: false
shore: true
lha: false
livery: "Syria (1)"
mission_types:

• BAI
• BARCAP
• CAS
• DEAD
• OCA/Aircraft
• OCA/Runway
• Strike
  pilots:
• Biff Tannen
• Marty McFly
• Doc Brown
• George McFly
  players:
• Joey Bagofdonuts
• Jimmy TheFoot
  '''

Keep in mind for now the blufor group will stay as is but i did create a custom harrier squadron. In a later section, I'll address the lack of F18 squadrons, a problem that is annoying many users of Liberation.

Adding Squadrons to Campaign

Take a look at the basic setup in the campaign yaml file we have now.

I created a new harrier squadron D:\dcs_liberation_5.0.0\dcs_liberation\resources\squadrons\AV-8BNA\VMA-621.yaml nicknamed "Old Gravy Legs".  I want to add them as an option to the Blue LHA (Tarawa) section.

'''yaml
Blue LHA:
- primary: Strike
secondary: any
aircraft:
- VMA-621
- primary: BAI
secondary: air-to-ground
aircraft:
- VMA-214
- VMA-223
- VMA-621
- primary: CAS
secondary: air-to-ground
aircraft:
- VMA-214
- VMA-223
- VMA-621
- primary: CAS
secondary: air-to-ground
aircraft:
- VMA-214
- VMA-223
- VMA-621
- primary: OCA/Aircraft
secondary: air-to-ground
aircraft:
- VMA-621
- primary: OCA/Runway
secondary: air-to-ground
aircraft:
- VMA-621
'''

If you look closely i added the OCA primary options and to each primary mission type, I added an 'aircraft' section. In the aircraft section, I specify the squadrons I want to be assigned. On most carriers, there is a squadron that focuses on air to ground more than air to air, another for EW, etc. So for SEAD perhaps only 1 squadron is meant to have expertise in that mission type. This is up to the author.

Rather than specifying a specific squadron, we can instead list an aircraft type.  For instance i will assign only Mig 21's for BARCAP missions on the redfor team out of Anderson AFB.

Andersen AFB

6:
- primary: BARCAP
secondary: air-to-air
aircraft:
- MiG-21bis Fishbed-N
- primary: BARCAP
secondary: air-to-air
aircraft:
- MiG-21bis Fishbed-N
- primary: Strike
secondary: any
aircraft:
- GRFS-001

Also note that for the "Strike roll" i specify squadron, for BARCAP i specify aircraft. Lets take a look at the bluefor carrier options. The biggest complaint i have seen with Liberation 5.0 is that people create a carrier campaign and then see only F14's available. They may own the F18 or have installed the A4 community mod but neither aircraft show up during the creation of the campaign.   If you're not sure what am talking about take a look at this  

Notice no A4, no FA18 squadrons available and this is using the USA 2005 faction. We fix this by adding the F18 or F18 squadrons to the campaign file.

'''yaml
squadrons:
Blue CV:
- primary: BARCAP
secondary: air-to-air
- primary: BARCAP
secondary: any
- primary: Strike
secondary: any
aircraft:
- F/A-18C Hornet (Lot 20)
- primary: SEAD
secondary: air-to-ground
aircraft:
- F/A-18C Hornet (Lot 20)
- primary: DEAD
secondary: air-to-ground
aircraft:
- F/A-18C Hornet (Lot 20)
- primary: BAI
secondary: air-to-ground
- primary: CAS
secondary: air-to-ground
- primary: Refueling
aircraft:
- S-3B Tanker
- primary: AEW&C
aircraft:
- E-2C Hawkeye
'''

At this point we have enough to start a campaign that uses F18's , F14's, harriers, my custom squadrons and even custom pilots. (Note i did not create a custom livery but this is not hard to add)

What's in Part 2 ...

short answer - the mission editor

In a previous post, I ran through a tutorial set where I built a Basic DCGAN. Similar to the PyTorch tutorial. Both use 64x64 images. I had some decent results but a 64x64 image isn't very useful and with upscaling I started seeing dilution of pixels.

Scaling up the image size to 128x128 or larger

What needs to be changed?

First, change the constants in the trainer, I'll go over why I needed to lower the batch size later. Make sure that your training data set is sized to match 128x128 or larger. This is done in a separate file included in the author's code.  

#trainer Constants
BATCH_SIZE = 16
IMAGE_SIZE = 128
FEATURES_DISC = 128
FEATURES_GEN = 128

Next, the Discriminator and Generator need to have layers added. The discriminator adds 1 Conv2d layer and the generator adds 1 Conv2dTranspose layer. This gets us from 64 to 128 on the generator. The author makes a "_block" section to actually create the conv2d layers (not seen in the snippet).

#DISCRIMINATOR CODE SNIPPET
self._block(features_d, features_d * 2, 4, 2, 1),
self._block(features_d * 2, features_d * 4, 4, 2, 1),
self._block(features_d * 4, features_d * 8, 4, 2, 1),
# added by me to sample up , i think this gets us to 128
self._block(features_d * 8, features_d * 16, 4 ,2,1),
# added by me to sample up , i think this gets us to 256
self._block(features_d * 16, features_d * 32, 4, 2, 1), 
# added by me to sample up , i think this gets us to 512
self._block(features_d * 32, features_d * 64, 4, 2, 1), 

self._block(channels_noise, features_g * 16, 4, 1, 0),  # img: 4x4
self._block(features_g * 16, features_g * 8, 4, 2, 1),  # img: 8x8
self._block(features_g * 8, features_g * 4, 4, 2, 1),  # img: 16x16
self._block(features_g * 4, features_g * 2, 4, 2, 1),  # img: 32x32
self._block(features_g * 2, features_g * 2, 4, 2, 1),  # img: 64x64  
self._block(features_g * 2, features_g * 2, 4, 2,1),   # i think 256x256
self._block(features_g * 2, features_g * 2, 4, 2, 1),  # should go 512x512

CUDA and GPU limitations

This is where a 'newb' like me will struggle. Going to 128x128 i already ran into memory limitations.

RuntimeError: CUDA out of memory. Tried to allocate 8.00 GiB (GPU 0; 11.00 GiB total capacity; 5.53 GiB already allocated; 4.45 GiB free; 5.53 GiB reserved in total by PyTorch) If reserved memory is >> allocated memory try setting max_split_size_mb to avoid fragmentation.  See documentation for Memory Management and PYTORCH_CUDA_ALLOC_CONF

This is a pretty common beginner error with Cuda. The most accepted solution is to lower your batch size. The computational complexity of the model is higher and the trainer is trying to allocate more memory than the graphics card has. Lowering the batch size should keep the trainer from trying to allocate as much memory on the graphics card at once.

To get to 128x128 I had to lower my batch size from 64 to 16.
To get to 256x256 I had to lower my batch size to 1

Workaround using CPU only

Let me start by saying, don't do this. My reasoning is that my desktop has around 48 gigs of RAM available on the motherboard whereas my 1080ti has 12 gigs. Attempting to get a 512x512 image this seemed to be my only option.

#device = torch.device("cuda" if torch.cuda.is_available() else "cpu")
# forcing the use of "cpu" even though cuda is available
device = torch.device("cpu")

This used up about 47 gigs of my 48 gigs of ram. My computer didn't seize but it was close. Mouse inputs and audio went out for a bit. The python code kept churning though and eventually failed out asking for more ram than I had.

Time for some reading

I am expecting a tutorial model to upscale to an unreasonable degree. I know StyleGan and StyleGan2 have good results on larger images and I may see what I can do with them. The issue is how much data is going into ram and I have to assume these other architectures have figured out a workaround.

Two promising papers:
Large Scale GAN Training for High Fidelity Natural Image Synthesis
Progressive Growing of GANs for Improved Quality, Stability, and Variation

This page does a good job of explaining "Brittleness" in Gan training
A Gentle Introduction to BigGAN the Big Generative Adversarial Network

Upscaling with "Bigjpg"

I looked at a couple of algorithms to use deep learning to upscale some images and stumbled on this program.  The results were interesting. The result using 4 different methods are shown below. The Bigjpg algorithm which uses "deep learning" does a decent job of unblurring the image at a larger scale.  The noise reduction looks like it smooths a bit too much.  
1 - Using Pillow library to 'resize' the original 64x64 image
2 - Using Bigjpg to upscale with no noise reduction
3 - Using Bigjpg to upscale with medium settings for noise reduction
4 - Using Bigjpg to upscale with hi settings for noise reduction

I read through and ran some of the items in "Generative Deep Learning". The book gets into the section about teaching the ML to paint, to write music, to write books. The paint section was useful. The book explains style transfer and the representational space.

I started researching a few other models and noting what they were used for. I'll try out styleGann next but for now I settled on a basic tutorial i found here https://levelup.gitconnected.com/painting-portraits-using-gans-with-pytorch-afeb69b1c5a1

The code is meant for abstract art style transfer and the generation of new art. It is using pytorch, keras, and tensorboard. The author of the code does a pretty good job of resizing and massaging the image data, all taken from Kaggle. The tutorial sets up tensorboard for monitoring and is a pretty good template for a GANN project. It has all the pieces in place.

Training First Epochs

I have ~11,500 images in the abstract folder. I resized them to 128x128 using the author's method. I noticed my GPU is not being used because i do not have a CUDA installation. So with CPU only, a batch size of about 64 , i should have around 180 batches per epoch.  It took about 5 to 6 minutes to run through a full epoch. Here are the generated vs. real images after a single epoch

We can start to see a transition from colorless shapeless blobs to a patchwork with color and more detail within only 5 epochs of training. For now I'll let it run and take a look later tonight while I fix my Cuda issues. CPU only is not a great way to train any deep learning model.

One Day later...

This was the final output after 100 epochs, honestly not that bad. (considering this was on a modern art data set).

Performance notes

I ran this program and trained the same data set 3 times.  
1 - (4.5 hours) Gaming Desktop - CPU only
2 - (16 Hours Laptop - 2cnd gen I7 CPU only
3 - (25 minutes) Gaming Desktop - CPU and single GPU (1080ti)

I will say that getting the CUDA GPU support to work was a bit of a hassle but worth it.  

Follow up and Scaling issues

These are some of the generated images and then the 'scaled' up versions using bilinear interpolation, sadly the interpolation can only do so much, and its not enough. Take a look