I tried upgrading one of our servers from Apache 2.2 -> Apache 2.4. It didn’t work so I purged the PPA but the configurations needed rolling back too (which ppa-purge command didn’t do). So, to completely remove Apache2 configuration files and reinstall them use the following command:
sudo apt-get remove --purge apache2 apache2-utils apache2.2-common
and then had to also reinstall php5:
sudo apt-get install --reinstall apache2-utils apache2 libapache2-mod-php5
Hope this helps someone get out of a pickle.