So I had some confusion about the notion of entropy cleared up today. It seems that it is the case that the notion of entropy is only defined up to the description of the physical system given, rather than being an intrinsic value of the physical system. So if I give you a box full of air, and ask you for the entropy of the box there is no value that you can tell me. The value of the entropy is relative to the description you give. So if one says it has a certain gas at a certain temperature and pressure, you might be able to compute the value of the entropy to be x. However, if one says that the box has these n particles, and for each particle one gives its position and momentum, then the value of the entropy will be 0, even though the box is the same in both cases.
Temperature suffers the same problem. Given the exact state of each particle in the box, the temperature is 0 or undefined or something. Because both temperature and entropy are given relative to the description, we can still compute in all cases the change in entropy to be greater than the integral of T-1 with respect to work.
Of course if temperature is only defined up to the description of the state, one wonders what thermometers measure. I suppose the answer has something to do with the size of the bulb.