Number of animals players can have/space issues?

H

Hare

Senior Member
How many animals do other sites usually allow players to have, and does the limit include offspring?  How many offspring do they have? 

What kind of restrictions are on players and their number of animals?

What techniques are used for coding and database for breeding games that save space? 

Leporidae (a rabbit breeding SIM) currently allows:
- A regular player to hold up to 150 animals.
- If upgraded, can hold up to 600 animals.

Rabbits can breed every 3 days and have 2-12 offspring per litter. Limits include offspring, so if a regular player has 50 rabbits, they can breed about 10-20 of them. If they have 100, they can breed about 5-10. Players are requesting that we not include offspring in the limits. This would bring a regular player's animal count from the restricted 150 to 1000+ potentially.

I considered making the offspring have 'temporary data' that takes up less space, but I'm not sure how to do that because they need to have genetics. The genetics take up a lot of space in our rabbit table. We store each allele as CHAR 5 because they can be things like A, cchd, En, etc. There's 36 alleles in total. We can't change it to integers without completely recoding most of the site.

I know I've asked about space-related issues before. I've taken quite a few pieces of advise and am still considering others like hosting our database on a different server. But I'm really curious about animals in particular and common practices or obstacles that other games face. 

 
Last edited by a moderator:
Novilar has a max limit of 200 pets currently per user. It is also quite a challenge to increase your territory to that size (so it feels rewarding).

Users can go over their limit, but are unable to breed any additional animals until they are under that limit. Offspring is included in the counts.

There are some additional strategies, but they are strategic, and have pro and cons. Some can implemented easily, others not so much.

If you are not constantly checking the genetic values of the pets, you can serialize and gzip them into a Blob field in the database. This can save some space, but you lose the ability to see that data without ungzipping and deserializing. However if this data is only looked at every 3 days, it might be worth it. Serialization and compression are costly, so this is really only a good strategy for data that you need to access often or you know you only need it on occasion.

Another way, if you are finding your database is the one struggling would be to perhaps cache of this information outside of the database. There are a boatload of methods or this type of thing, so not knowing more specifics, I couldn’t recommend anything in particular. Caching your pages strategically or even doing some query caching for common queries can provide a boost also.

 
Thank you, Digital! We've considered turning the genetics into a blob. We constantly view and update genetics, so that may not be an option. I can look into caching. 

And thank you for the info on Novilar's limits. I've been looking for any sites that have higher limits on allowed number of animals for basic players. I haven't found any that allow significantly more yet (i.e. in the thousands) but am still looking around.

 
Hare said:
Thank you, Digital! We've considered turning the genetics into a blob. We constantly view and update genetics, so that may not be an option. I can look into caching. 

And thank you for the info on Novilar's limits. I've been looking for any sites that have higher limits on allowed number of animals for basic players. I haven't found any that allow significantly more yet (i.e. in the thousands) but am still looking around.
Click to expand...
http://wern-ancheta.com/blog/2014/01/26/getting-started-with-caching-in-php/

Could be a useful read on caching strategies. There are few different approaches, so have to do some digging to find one that works, or a set of them that provide you benefit.

 
I do something on my site called a license. It limits how many pets you can have at a time and each bigger license costs more to buy and some are only available as upgrades. In your case, I would keep a string of the pets genetics so you only have to calculate them when they breed since you allow them to breed so many or keep a single database with all genetic possibilities as they're discovered and assign that to each pet so you only have one copy of each genetic possibility rather than millions that have duplicated records for each rabbit. Having a good indexes will also help with the db speeds or you can use in memory tables for breeding. There are a lot of possibilities to help you out. Also once they pick the babies they want delete the rest and optimize the table every night. 

 
Thank you everyone for the advise. I have looked into many options and spent many hundreds of hours recoding the basic structure of the site and optimizing the database. I recently made another huge optimization.

unfortunately, it's not enough. Right now, we have a thing called 'legacy accounts' that allows old players to keep thousands of rabbits because those were the site's original limits (I regret this now). I want to eliminate the legacy's, but it's too late. The staff and players say it will ruin the site if the legacy players are reduced to the normal limits (150 basic and 600 upgraded). 

The site can't handle this though. We lost about 1.5GB in the last few weeks from gaining 10k rabbits in the database. This means that it would take about 20 legacy account holders coming back online and having 1000 rabbits each to cause the site to crash from lack of disk space. It would take 200 basic players players the same if they have more like 100 each. 

I've done all I could. I can't seem to fix it, been trying to fix these space issues for ages. Most of the rabbit data is INT to save space and I keep as few backups as possible. Maybe there is something else I'm doing wrong, I don't know. I've hit a dead end with this site though. I love it dearly, but it's going to crash and die at this rate.
 

A word of caution to anyone making a new site. Don't allow players to have thousands of animals before knowng how much space this takes up and having a longterm plan for your resources. 

 
Last edited by a moderator:
You say that most of the rabbit data is INT in order to save space - do you have a need for such high numbers?  Or would smaller data types do you better?

I would love to help if you have anything that I can help with - I have been working with larger data sets for a while now (doing point of sale analytics and reporting).

~judda

 
Sounds like something is not right.  The genetic data shouldn't be taking up that much room in the database.  Khimeros has a million+ pets with genetic data and that data doesn't take up much room.  It's the images that take up massive space on khimeros.   But from what I know of your game, most of your pets have static images yes?  Every pet doesn't have its own separate image that needs to be stored?  I can't see why your users shouldn't be able to have hundreds of pets without a problem if all the data is being stored properly.  I'd get Judda to check that out for you.

 
judda said:
You say that most of the rabbit data is INT in order to save space - do you have a need for such high numbers?  Or would smaller data types do you better?

I would love to help if you have anything that I can help with - I have been working with larger data sets for a while now (doing point of sale analytics and reporting).

~judda
Click to expand...
Thank you for the offer. What datatype can I use instead of INT? Here is a screenshot of the data for one rabbit's genetics. Each column has a number for the allele.

View attachment 2350

And then we display the alleles as letters (which are specified in another table). I recently switched these over from CHAR.

View attachment 2349

The images are rendered based on the genetics using imagemagick and are not stored in the database. It finds out whether a rabbit has Aa or aa (for example) with a code I INCLUDE. The genetics determine which layers are turned on, etc. No images stored in the database except for the layer pieces for each breed. 

The rabbit data has 74 columns per rabbit, most are INt but some are DECOMAL, DATE, CHAR, and VARCHAR. There are a few other tables with data associated with rabbits such as events, show results, and some other that take up some space.

The rabbit table itself is only about 24 MB (used to be about 29 MB before I switched the genetics from CHAR to INT). The database as a whole is about 150 MB. We keep 2 backups of the site, one daily and one weekly. We lose about 1 Gb every 5-10k rabbits in the DB, though (when I check up on the disk space usage).

 
Last edited by a moderator:
Are you on Discord?  It may be easier to discuss there.

For data types, if you are just doing on/off - TINYINT(1) would work.  Even with multiple options, TINYINT may work better for you.

https://dev.mysql.com/doc/refman/8.0/en/storage-requirements.html

I'm a little confused about what you are saying.  You say that the database as a whole is 150 MB but for every 5-10k rabbits, it grows 1 GB.  This tells me that the issue isn't with your database but with the actual images that you store for the rabbits?

 
judda said:
Are you on Discord?  It may be easier to discuss there.

For data types, if you are just doing on/off - TINYINT(1) would work.  Even with multiple options, TINYINT may work better for you.

https://dev.mysql.com/doc/refman/8.0/en/storage-requirements.html

I'm a little confused about what you are saying.  You say that the database as a whole is 150 MB but for every 5-10k rabbits, it grows 1 GB.  This tells me that the issue isn't with your database but with the actual images that you store for the rabbits?
Click to expand...
Yep I have Discord, Lunar Eclipse#6934
We need the data to dintinguish between about 100 or less different possibilities (rght now we have allele #0-72).
We don't store images so I'm rather baffled by that too. 

 
Last edited by a moderator:
It sounds like your data issues are disk space related and not necessarily db related. If your site is generating a huge number of errors in your apache error logs that is often why you run out of disk space so quickly and why it happens when the rabbits are being generated means you're probably getting errors or warnings when they are made that are inflating your logs if you generate them every time instead of storing a static image. On your server cd /etc/logs/apache2 and then type ls -la and copy/paste a screenshot. My guess is this is your actual culprit! 

 
Last edited by a moderator:
Thank you, Design1online. I couldn't find any logs under etc/apache. There is a logs folder and a 207 mb log file. There's also a cache file that's about 600 mb. The error logs on the site used to have issues with rendering (it used to shoot up 20 mb a day), but I had fixed the error and we no longer have any image errors going into that log (it doesn't grow much). 

Judda helped me search through our files to find the big ones. Not sure what it is still, but I'm keeping an eye on it to see how it grows. Might be a number of things adding up. 

I found a new owner to take over the site, since I can't afford it with the space issues it has. He says he'll be able to foot the bill for it from now on, so I'm really relieved. I'd like to still figure out what's going on even though I don't own the site anymore.

 
Last edited by a moderator:
An update on this: 

I got a lot of help and am tremendously grateful for that and for the assistance from my friend. I thought I was not going to be able to keep the site anymore, but he took it off my hands and fixed the issues we were having. We're now back to working on it together.

It was a culmination of a few larger files and their backups, including an image cache and some other cache files. I've since deleted those, freeing up enough space so we're no longer on the brink of downtime.

We also reduced the animal limits and it has made a world of difference. 

 
Last edited by a moderator:
You must log in or register to reply here.
Back
Top