Laws related to privacy and photography can vary significantly depending on the country and jurisdiction you are in. In many places, a person generally has a reasonable expectation of privacy within their own yard or property.
Taking photos of someone in their yard without their permission could potentially infringe on their privacy rights, especially if the photo is taken in a way that invades their privacy or captures them in a private moment. However, the specifics of what is considered legal or illegal can vary based on local laws and the circumstances surrounding the situation.
In some regions, laws may allow for photography of individuals in public spaces where there is no expectation of privacy, even if it includes their yard as long as the photographer is standing in a public area while taking the picture.
If you believe your privacy has been violated by your neighbor taking your photo without your consent, it's best to consult with a legal professional who can provide advice specific to your jurisdiction and the local laws that apply. They can help you understand your rights and options in such situations.