+Vivek
I don't know the exact bug here, but one possibility is that since you've disabled the auto-extend, that it's leaving the metadata size at it's default? There is indeed a relationship between the two and I did a lot of testing around it to make sure we were not going to hit what you've hit :-/ At that time however, we did not have the auto-extend feature.
So I think you may have found a situation where we need METADATA_SIZE=N but you said you set that at 2%?
Can you paste the output of "docker info" and "lsblk" please?
From April 2015 when I last looked at this, the tests ran were to start 1000 httpd containers and measure the thinpool used size. Here:
So, for 1000 rhel7+httpd containers, on-disk it was using about 3GB (left-Y-axis).
And the metadata consumed was about 50MB (right-Y-axis).
You can see we concluded to set metadata @ 0.1% of the data partition size, which in the case:
I am wondering what your docker info will say.
On Fri, Jul 22, 2016 at 8:08 AM, Andrew Smith <me andrewmichaelsmith com> wrote:
HiWe've been using docker-storage-setup (https://github.com/projectatomic/docker-storage-setup) to set up our CentOS LVM volumes.Because our block devices are fairly small and our containers don't write data inside them (and we hit issues with resizes being very slow) we have a static setup:AUTO_EXTEND_POOL=FalseDATA_SIZE=98%VGThe intention is to just create the lv as big as we can up front, because we don't need that space for anything else. We leave 2% for metadata.However - we're running in to a problem, where our metadata volume is hitting 100% before out data volume gets close (sometimes, 60%). We're deploying kubernetes on these boxes, which garbage collects based on data usage not metadata usage, so we are often hitting the 100% and taking outboxes.What I'm struggling to understand is how much metadata a docker image or docker container uses. If I can find that out I can figure out a good data:metadata ratio. Is there a way for me to discover this?Or, as I suspect, am I missing something here?Thanks--Andy Smithhttp://andrewmichaelsmith.com | @bingleybeep