We investigate a significant model-observation difference found between cloud-base vertical velocity for continental shallow cumulus simulated using large-eddy simulations (LES) and observed by Doppler lidar measurements over the U.S. Southern Great Plains ARM Facility. The LES cloud-base vertical velocity is dominated by updrafts that are consistent with a general picture for convective clouds, but is inconsistent with Doppler lidar observations that also show the presence of considerable downdrafts. The underestimation of simulated downdrafts is found to be a robust feature, being insensitive to various numerical, physical, or dynamical choices. We find that simulations can more closely reproduce observations only after improving the model physics to use size-resolved microphysics and horizontal longwave radiation, both of which modify the cloud buoyancy and velocity structure near cloud side edges. The results suggest that treatments that capture these structures are needed for the proper simulation and subsequent parameterization development of shallow cumulus vertical transport.