Some giants are larger than others, and their babies are appropriately sized.
You can do some rough estimates, with human babies weighing on average 7 pounds and adults 140, so a weight of about 5% of the adult giants weight should be appropriate.
As far as dimensions, a baby's avg height is about 20 inches, which is less than third of the avg human height of 66 inches.
So, a 20 foot tall giant should have a baby about as tall as a human, give or take, though likely considerably heavier.