Try using the method setPreferredSize(Dimension) for objects you want
to control.

JPanel exitPanel = new JPanel(new FlowLayout());
exitPanel.setPreferredSize(new Dimension(200,100));
exitPanel.add(exitButton);
getContentPane().add(exitPanel, BorderLayout.SOUTH);

Placing JButtons on a panel  using FlowLayout shows the buttons in
their preferred size, rather than using the whole panel or area.