Remove the meat from the cooked lobster and cut into fine dice. Remove the tomalley and reserve. If there is any coral, add it to the tomalley.
Roll the crackers free and combine with the butter, tomalley, salt, and cayenne. Blend into a smooth paste. Bring the milk to a boil in a heavy saucepan, stir in the paste, and add the diced lobster. Bring to a boil again and serve at once.