memCron 0.3

memCron 0.3 is finally ready and there are several new features added to this new version:

  • Customizable chart formatting — pick the data you want to plot! (see demo)
  • New parameter “$downsize_resistance” introduced to prevent multiple resizes during short-term, rapid memory fluctuations

With the addition of this new parameter, memCron is now more stable than ever. If you have experienced multiple memory resizes by memCron within short period of time in previous version, memCron 0.3 has it all solved.

This should fix the “exceeding 30 resizes per day” problem that could happen in previous versions. Now, regardless of how rapidly your server’s memory usage fluctuates, memCron has you covered!

memCron 0.3 is a must have whether you are a current user or are new to memCron. The way how memCron handles your server resources will definitely change the way how you work with DreamHost’s manage resources panel up till now.

Wait no longer, download and install memCron 0.3 to experience the difference!

Download: memCron 0.3 (30.36 KB)

32 thoughts on “memCron 0.3”

  1. This looks awesome and have come to find out about it via Dreamhost announcement on their blog.

    I notice that it’s currently at version 0.3. Does this imply that the product is in beta/unstable?

    When will the product be 1.0 release ready?

    1. Hi Tim, thanks for informing me about the announcement on DreamHost blog, I can’t believe I won! Wow, this is so awesome!

      Regarding your concern, I believe memCron is stable enough right now for most users. However, there could be some downtime while you experiment with the different configurable options that memCron offers. The default value should work well enough but I haven’t had much feedback from actual users so far. As more people start using this script and actively give me feedback, memCron should become more and more stable over time.

      Version 1.0 will be released when the integrated cron job and web admin panel features are completed. When will it be? I’m not sure, if the feedback is overwhelming I will try to get it all working asap.

  2. great tool, trying it out as i type this.
    i actually did a similar thing just as a shell script and therefore setting the values in the config.cfg felt oddly familiar. :)

    one config value i’m really missing though and that is a hard cap for the maximum memory amount that may be set by memcron. because if something nasty happens (like a homebrew script gone wild), it will happen when you are on a two week vacation and aren’t checking your server stats.

    so, currently the hard limit is the max memory available by dreamhost.
    4gb = $400/month.

    1. Hi capsized, for the hard cap you suggested, is it a limit that you manually set just so memCron won’t request memory that are higher than this limit? Right now memCron will request up to 4000mb of memory as needs arises, is that too risky for you because of the $400/month bill?

      If a hard limit lower than 4000mb is set, you are preferring downtime over larger bills, that’s fine?

      1. That’s a value I’d be interested in as well. As I am working on a student project with a limited budget and do not want to risk spending too much.

          1. @Yaosan

            Any chance that you’ll add a feature to notify you via email if your PS instance exceeds X amount of memory?

            Pretty please :)

          2. May I also suggest to add a configurable minimum limit or some other means to tune the “server is idling” behaviour as well?

            My server reduces its resources too much while idling over night to cater for the first avalanche of visitors in the morning and drops completely into a series of “500″ then. Subsequently, this apparently causes a Dreamhost-triggered watchdog to automatically reboot the PS instance, and so the inital ramp-up in the mornings is quite a bumpy ride.

            I’d rather have a bit of security margin while the PS operates in the low memory range while I don’t feel the need to increase $mem_free_to_used_ration in general.

  3. I played with memCron last night and VERY awesome!!!

    Just a few feature requests:

    * hard cap on max memory allocated – yes, this will cause your server to crash but save you $$$

    * email notifications if memory allocated exceeds X (use case, if you know your server shouldn’t be exceed X memory – if it does, it might indicate an orphan process stuck consuming memory and maintenance needs to be performed)

    Also, what’s this about “integrated cron job and web admin panel features are completed”? Do you mean integrated somehow into Dreamhost Panel? If that could be done, that would be AWESOME. I’d much rather have my graphs accessible via the Dreamhost Panel than on my domain

    1. Hi Tim, thanks for trying the script out and giving me feedback.

      Hard cap on max memory will be implemented in the next version due to popular demand, sounds like a good feature to have for those who are worried about enormous bills, even though it’s highly unlikely to happen. :P I guess it never hurts to be on the safe side, right?

      Email notification is another great suggestion, so I believe this threshold for email notification will be different from the max memory cap? I can probably put in a list of current running processes within the body of the email, that might be helpful for finding the culprit process.

      The integrated cron job and web admin panel has nothing to do with DreamHost Panel. So far DreamHost doesn’t have an API to insert menus into their panel, so what I mean is adding an admin page in later versions that you can log into and change all memCron settings directly within your web browser. This admin interface will update the config file used by the memCron script and also set up a crontab properly based on your $cron_interval argument.

      1. @Yaosan

        You’re awesome.

        To answer your memory notification question, yes – it would be great to be able to separately define what the memory threshold amount should be … not related to the max memory feature.

        Though how I would use it, but many others might find a different use is to do something like:

        $max_memory_allocation = 2000 // don’t allocate more than 2GB of RAM (could case server crash) – if not defined, allocate as much RAM as Dreamhost allows

        ——————–

        $threshold_notification_memory_amount = 1500 // email me what current processes are taking up 1.5GB (or more) of RAM
        $threshold_notification_email =

        Of course, there is probably better config parameter names but you get the idea.

  4. The settings below have worked perfectly for me. I’m sure I could edit them even more to make it work even more efficiently but I haven’t had the time lately to test it.

    I have over 10k uniques a day with avg 10x views/unique. I’m running various software that use dynamic pages, so my memory usage changes quite a bit throughout the day but with these options (and proper tweaking of the software) I can keep the memory set to 150mb for most of the day and I won’t have any crashes because memCron does its job so well jumping it to 200mb-280mb when I need it and dropping it back down when it’s done.

    With these settings I do about 10-20 changes throughout the day.

    $cron_interval = 5;
    $mem_free_confidence = 0.999;
    $mem_free_to_used_ratio = 1.1;
    $downsize_resistance = 9;
    $num_data = 80;

    1. Thanks for sharing your feedback John, it’s great to see memCron working for different users. The more feedback I receive the more confident I am with the scaling algorithm I implemented, so thank you!

      For those who would like to tweak memCron for maximum efficiency, as can be seen in John’s example, $mem_free_to_used_ratio and $downsize_resistance is the two option that you should play with.

      Lowering the former lowers your bill but increases the chances of you seeing “internal server error” when your server runs out of memory during spikes. Lowering the latter gives memCron more room to fluctuate but this will also cause memCron to use up more changes per day. If memCron is making less than 20 changes per day for your server right now then you have some room to lower your $downsize_resistance.

      Tips: you can use the “downsize count” plot option to help you decide what is a good value to set $downsize_resistance to. From such a graph, you can get a rough idea how many extra changes will be made if the resistance count is lowered from say 12 to 10.

  5. I also like the hard memory cap and e-mail notification suggestions. I also have an application where I’d like a low memory restriction because in this application, demand grows faster than memcron can raise the limit, so I like it not to go below 300MB. I actually added it myself in v0.2, so I know it’s not a difficult option to add.

    Anyway, memcron is mostly reducing my bill because I don’t have to keep the max memory available all the time. Awesome tool! Thanks.

    1. You can adjust the memory allocation in both Dreamhost PS Manager and memCron, so you could get them to match if you tweak the memory management settings.

      While Dreamhost PS Manager has Twitter, memCron has a gui page to graph the usage for you and includes CPU usage. For me, this is more useful than a tweet.

      I really appreciate the effort that has gone into making this work so well. I mainly work with Windows, so writing something like this myself would have been a real challenge. Thank you so much for a great free product.

      1. Thanks Bret, I’m glad memCron is working great for you. You’re right that the memory setting can be tweaked to reserve more memory. For memCron, the parameter you should look into is $mem_free_to_used_ratio.

        With that said, I’m pretty confident that the default value of 1.2 should work well for most users. With a free to used ratio of 1.2, it will take a memory spike that raises your memory usage 2x within 5 minutes to bring down your server. For those who want more safety margin, feel free to increase the ratio but that extra margin do come with a price. :P

        btw, if you somehow reduces your free to used ratio to save some server cost, the $mem_free_confidence will kick in when the standard deviation increases due to a spike. So yeah, you’re double protected by both parameters, pretty much. ;)

    1. The nature of this script requires me to do some thorough testing before releasing any new features, and that alone will take a few days at least.

      My busy job allows me very limited amount of time to work on this project. I’d like to release as quickly as possible but I don’t want to release a half-ass product and have users suffering from it because any bugs can potentially brought down their whole PS when not enough memory is allocated.

      Anyway, I hope I can get the next version released sometime next week but things could still change so don’t put your hope too high in case the release date is pushed back.

  6. Just an update on the progress. I manage to work extra hours yesterday and I think I have the min, max cap and threshold setting working. Now I just need to run the test on my own PS and if everything works smoothly I’ll have the new version released this weekend. :D

  7. This is good. I just started using it today and like it so far.

    It looks like the two features (email when mem increases some threshold and max memory to allocate) I was interested in have already been coded for the next version which is great.

  8. Something strange is happening. The graph is showing the total memory as 538MB while Dreamhost panel (as well as the ‘free’ command) show it as 626MB. Why the discrepancy?

    1. Hi Zack, the discrepancy is due to a memory sizing bug during downsize operation. This bug has been fixed in the latest version, which will be release either later today or tomorrow.

      Thanks for your bug report, they’re always helpful to make memCron better and better. ;)

  9. Just a note to let you know that this app is great. It should really be in the DH panel, I hope they buy it from you and make it part of their core services.

  10. I got this error message, how to deal with it?

    501 Protocol scheme ‘https’ is not supported (Crypt::SSLeay not installed)Aborting at /home/lvya/eswnman.com/memcron/memcron.pl line 235.at /home/lvya/eswnman.com/memcron/memcron.pl line 235

Leave a Reply

Your email address will not be published. Required fields are marked *

You may use these HTML tags and attributes: <a href="" title=""> <abbr title=""> <acronym title=""> <b> <blockquote cite=""> <cite> <code> <del datetime=""> <em> <i> <q cite=""> <strike> <strong>