glStencilMask controls the writing of individual bits in the stencil planes. The least significant n bits of mask, where n is the number of bits in the stencil buffer, specify a mask. Where a 1 appears in the mask, it's possible to write to the corresponding bit in the stencil buffer. Where a 0 appears, the corresponding bit is write-protected. Initially, all bits are enabled for writing.
GL_INVALID_OPERATION is generated if glStencilMask is executed between the execution of glBegin and the corresponding execution of glEnd.