Microsoft seems to be having a devil of a time with the password feature of the dial-up networking system. Normally you should be able to put your name and password into the dial-up networking box, check the 'save password' option, and never have to worry about it again. This isn't always so.
There is a bug in the password database function which can lead to corruption of the encrypted password file stored on your hard drive. This file, if not readable will not allow saving the new password...thus you'll have to put in the password each time you run dial-up networking.

The fix, found on the Microsoft Knowledge Base, is to delete the associated .PWL file, re-boot your computer (you may receive that 'log onto network' box when re-starting, just enter your name and a blank for your password), then go back into your Network icon (in Control Panel) and make your default login Windows instead of Microsoft Network. The next time you use the Dial-Up Network you will have to re-enter the name/password and it should properly save it this time. There is also a 'fix' available by clicking HERE that will take care of this in the future.

Take a look in your \WINDOWS sub-directory for files called *.pwl. You should have one for the affected login name, just delete it and it will be re-made when the dial-up program is run.

UPDATE - UPDATE - UPDATE
You say Windows 95 won't remember your dial-up password? It might be because you installed the Windows 95 Service Pack 1. Microsoft admits that this update has a bug that messes with your password cache.

To fix the problem, point your Web browser to http://www.microsoft.com/windows95/info/passwd.htm

and download the Windows 95 Password List Update (and read all about it, if you care to). Once the download is complete, double- click on mspwlupd2.exe to install the necessary files.

(Tip: To ensure that all corrupted files have been removed from your system, delete any Password Caches [all PWL files in the Windows folder] before installing this update.)