The MSDN guidelines for standard exceptions states:
  Do use value for the name of the implicit value parameter of property
  setters.
  
  The following code example shows a
  property that throws an exception if
  the caller passes a null argument.
public IPAddress Address
{
    get
    {
        return address;
    }
    set
    {
        if(value == null)
        {
            throw new ArgumentNullException("value");
        }
        address = value;
    }
}
Additionally, the MSDN guidelines for property design say:
  Avoid throwing exceptions from
  property getters.
  
  Property getters should be simple
  operations without any preconditions.
  If a getter might throw an exception,
  consider redesigning the property to
  be a method. This recommendation does
  not apply to indexers. Indexers can
  throw exceptions because of invalid
  arguments.
  
  It is valid and acceptable to throw
  exceptions from a property setter.
So throw ArgumentNullException in the setter on null, and ArgumentException on the empty string, and do nothing in the getter. Since the setter throws and only you have access to the backing field, it's easy to make sure it won't contain an invalid value. Having the getter throw is then pointless. This might however be a good spot to use Debug.Assert.
If you really can't provide an appropriate default, then I suppose you have three options:
- Just return whatever is in the property and document this behaviour as part of the usage contract. Let the caller deal with it. You might also demand a valid value in the constructor. This might be completely inappropriate for your application though. 
- Replace the property by methods: A setter method that throws when passed an invalid value, and a getter method that throws - InvalidOperationExceptionwhen the property was never assigned a valid value.
 
- Throw - InvalidOperationExceptionfrom the getter, as you could consider 'property has never been assigned' an invalid state. While you shouldn't normally throw from getters, I suppose this might be a good reason to make an exception.
 
If you choose options 2 or 3, you should also include a TryGet- method that returns a bool which indicates if the property has been set to a valid value, and if so returns that value in an out parameter. Otherwise you force callers to be prepared to handle an InvalidOperationException, unless they have previously set the property themselves and thus know it won't throw. Compare int.Parse versus int.TryParse.
I'd suggest using option 2 with the TryGet method. It doesn't violate any guidelines and imposes minimal requirements on the calling code.
About the other suggestions
ApplicationException is way too general. ArgumentException is a bit too general for null, but fine otherwise. MSDN docs again:
  Do throw the most specific (the most derived) exception that is
  appropriate. For example, if a method
  receives a null (Nothing in Visual
  Basic) argument, it should throw
  System.ArgumentNullException instead
  of its base type
  System.ArgumentException.
In fact you shouldn't use ApplicationException at all (docs):
  Do derive custom exceptions from the T:System.Exception class rather than the T:System.ApplicationException class.
  
  It was originally thought that custom exceptions should derive from the ApplicationException class; however, this has not been found to add significant value. For more information, see Best Practices for Handling Exceptions.
InvalidOperationException is intended not for when the arguments to a method or property are invalid, but for when the operation as a whole is invalid (docs). It should not be thrown from the setter:
  Do throw a System.InvalidOperationException exception if in an inappropriate state. System.InvalidOperationException should be thrown if a property set or a method call is not appropriate given the object's current state. For example, writing to a System.IO.FileStream that has been opened for reading should throw a System.InvalidOperationException exception.
Incidentally, InvalidOperationException is for when the operation is invalid for the object's current state. If the operation is always invalid for the entire class, you should use NotSupportedException.