Everything in Guice is bound immediately. You cannot specify something like lazy binding. But what you can do is not binding to the desired object itself but to the provider of the object:
@Inject private Provider<Client> clientProvider;
By this way you can choose when to retrieve the desired object when need at not at object instantiation.