One of the few drawbacks of WordPress is that it loves to use a lot of memory.
This can become an issue when the memory allocation for PHP is low (often 32mb on various hosts).
Although some WordPress installations may not be affected some will be, due to things like resource hungry plugins and widgets being freshly installed.
When this issue occurs, you will be greeted by the unfortunate message somewhere along the lines of:

Fatal error: Allowed memory size of 33554432 bytes exhausted (tried to allocate 7680 bytes) in some/random/file/in/wordpress.php on line 123

This error message will most likely surface when you are trying to access a plugin or other feature within your WordPress admin area, and could have one of many various files listed such as wp-settings or something similar. The file shown in this error message is generally irrelevant to solving the issue at hand.

Editing the php.ini file

Editing the php.ini file will in most cases solve the problem. The php.ini file is used to declare changes to your server’s default PHP settings, and will usually succeed in changing the server’s memory allocation to PHP.

Troublesomely enough, the php.ini file can be located in different places on the server depending on the web hosting company you are using. You can usually access it using FTP, but sometimes it may be non-existent. Your best bet is usually to search through all the folders in the root folder of your website (not WordPress folders) such as ‘etc’ or ‘php’, but its best to try everything just in case.
If you do happen to find your php.ini file, you can download it to your computer, open it using a text editor like notepad and add the following line to the page:

memory_limit = 256M ;

If ‘memory _limit’ already exists in the file, simply increase its value. 32m, 64m, 128m and 256m are all acceptable options.

Creating a php.ini file

If you are one of many who cannot find your php.ini file, you will have to add one to your site. You can create your own one using a text editor by adding the line

memory_limit = 256M ;

and saving as php.ini, or alternatively download a pre-made one here (right click, save as).
Upload the file to the wp-admin folder of your website and test to see if the problem has gone away!

Side note

Unfortunately, some hosting services do not allow for the memory allocations to be overwritten, and if this is the case, the above fix will not work resulting in you having to call tech support for assistance :)