memCron has been updated to address the 2x increase of swap memory recently implemented by DreamHost. So far, there hasn’t been any official confirmation of this change. I’m guessing DreamHost is still experimenting with this new change and doesn’t want to make any commitment to support these extra swap memory just yet.
Here are the changes I’ve made in this minor update:
- Added: favicon for easier recognition of memCron tab in browser
- Fixed: Memory calculation due to the 2x increase of swap memory by DreamHost
- Fixed: Proper max axis value for “Used + Total Memory”
Download: memCron 0.4.1 (32.57 KB)
Please give this update a try and let me know if there’s any issue. I have tested this on my own PS server and it seems to work just fine.
{ 141 comments… read them below or add one }
← Previous Comments
yeah memcron is NOT working properly for me anymore.
it sets the memory to around 2427 MB when I’m only using around 300-400mb on the DH panel.
I had to disable it.
From DreamHost customer support:
Sorry about the troubles you’ve encountered. As for the weird memory readings you’re reporting, there’s actually a valid, albeit weird reason for this. Previously, we had no way to track cache usage on a per-guest/PS basis. With a new kernel update that’s rolling out, it can now be accurately reported by the guest. In light of this, the guest’s kernel now reports cached memory as well, where before it was always 0.
Now this is being included in the free counts, yet this cached memory can be reclaimed by the system at any time.
Currently, all memory readings do look fairly accurate for the output of ‘free’ in light of the changes:
ps1****:~# free -m
total used free shared buffers cached
Mem: 912 258 653 0 0 0
-/+ buffers/cache: 258 653
Swap: 0 0 0
As for your reference to
http://memcron.com/2009/08/memcron-0_4_1-fixes-memory-issues/#comment-342
yes, all PS guests are under a trial to get rid of the swap reporting, since in reality, SWAP doesn’t really exist on a PS but is instead managed by the host. As for how the numbers now work out, I’m afraid I can’t really comment too much on that as well nothing’s been really announced yet.
And now DH support just emailed me again saying there’s something wrong with the mem usage being reported on my account. They’ll “get back to me” *sigh*
Here is an updated version of memcron.pl* that calculates memory usage by summing stat information from each individual process. It also takes into account the new kernel by ignoring swap when the kernel is reporting cache. I hope it helps those who were getting wild numbers.
* This isn’t my new version with the separate collector daemon.
Hi Ben, I replaced my memcron.pl with your source code but my memory is still resized to 4000 MB, following are the command result:
Total: 450 Used: 429 Free: 21
Mean: -3602879701896090 Stdev: 7205759403792756.00 Load: 0.06
Mem: 150 Swap: 300
Tolerance: 13667009612242782 Tolerance (free): 22267470489563948
Target: 4000
Memory was resized from 150 MB to 4000 MB!
API Response:
success
memory_mb
4000
Sorry, I noticed a typo in that one. Try this: memcron.pl
This one is working well for me. It is reporting very close to the same numbers I see on DreamHost’s resource usage charts. Tomorrow, when it’s no longer taking the corrupt history into account, I should be able to let it start managing memory again.
Thank you!
This one should get even closer to DH’s numbers: memcron.pl
Also, your logs are likely corrupt with those huge negative numbers. I would suggest removing the .csv and .log and starting over.
Thanks, Ben
I followed your advise and grabbed the latest source code (http://memcron.com/2009/08/memcron-0_4_1-fixes-memory-issues/#comment-374) and now the chart looks like back to normal.
I am still monitoring it and hopefully DH doesn’t change anything drastically again.
The problem started a few days ago, my memory usage was creeping up above normal. And since yesterday, it always jumps to 4000 MB. Now I have to disable it.
Is there anyone has a workaround for this?
Any updates on the memcron product?
Hi,
Just installed this and it doens’t seem to do any changes to the memory. I get times where it has blank spots in the graphic. Example here:
http://skitch.com/bmoyle/nwpyp/resources-usage-for-ps14962-memcron
Any idea what I have done wrong?
I also disabled memcron for the time being, as it was way off. I contacted DH support and now I understand why. I will try that .pl you link to when I have the chance.
I don’t want to tell you how to run your site, but you should put a notice on the front page here, as I stopped by and when I saw there were no new posts since last year I figured the problem was not with memcron.
Regardless, I think memcron and DreamHost PS are great. The more I use them, the more I like them.
I’m not getting any email notifications when memory goes past the threshold. Must I use my domain name branded email? I wanted to put in an email that would be sent to an email account that would push to my phone.
I think we need to add a way to drop one or two spikes off the memory usage before adjusting memory like psmanager does. It doesn’t need to adjust memory way upward for a one-time spike in usage and then wait an hour before it can downgrade it again. I might look at that myself next week if no one else wants to tackle it. And then, the internal processing in memCron is handled differently so it may not even be possible. Time will tell.
My new version (which is so far off of memcron that it’s hard to call it that) uses different algorithms to handle noise as well predict the ebbs and flows throughout the day.
I’ve got it tracking the memory usage of daemons, scheduled (e.g. cron/at), and interactive processes separately and based on that can make some (usually valid) assumptions about whether a spike in memory usage is transient.
Been wanting to get it released for a few weeks now, will try for next.
That sounds great actually. I’ll just sit tight and wait for Ben’s new memory manager.
And now I have another problem. Yesterday I was trying to figure out why the memory was being set to the max from time to time, so I put code in memCron to set the downsize_count to -1 right after the call to the API to set memory. Today, I’m looking at the log and when it jumps to max, the number is not -1, so apparently, something else is adjusting the memory. Time to create a new API key and revoke the old one I guess. I wish you could add a comment to the adjustment so I could see WHAT was doing it.
OK… maybe it IS setting the memory. Looking more closely, there are generally 300 second gaps between log entries, and where the memory spikes up, there is 600 seconds, so perhaps it’s crashing before it can log the result.
Sigh.
OK… maybe it IS setting the memory. Looking more closely, there are generally 300 second gaps between log entries, and where the memory spikes up, there is 600 seconds, so perhaps it’s crashing before it can log the result.
Sigh.
Now I see. It’s probably getting a bad return from the API call, at which point, it dies rather that alerting anyone. This should be sending an email to alert someone if it can. Certainly it shouldn’t be calling die at this point unless you’re debugging.
Ben-
Thanks a ton for posting that fix, greatly appreciated. I noticed my last DH bill was unusually high, but didn’t really think anything of it. It wasn’t until I came here that I realized there was a problem with the script. I installed your 0.5.3 version last night and I can already see the difference this morning. The memory slider is much closer to where it needs to be than before.
I’m looking forward to the new version you’ve mentioned in Feb. 26. Can we officially call this a fork of the original product? If so, I would be happy to help you set up a website to host this project — just let me know. It certainly deserves more attention than to just be buried in the comments of this blog post
Thanks again for your hard work!
-Richard
I can’t find Ben’s fix anymore – it seems that the pastebin page no longer exists. Can you please post it again?
Here is the 0.5.3 version by Ben:
http://ben.pastebin.com/f46b981bd
Not sure if there is an even newer version, but this is the one I’m using and it’s working quite well.
← Previous Comments