You mention that you are not that old. What does that mean? I am old but my memory issues began when I was around 49, which to me is not old. And I hated it. Over the years I have worked very hard on this and found out a number of things. One is that sometimes it is a medical thing in that certain deficiencies worsen this problem. A lot of vegetarians who are not careful have vitamin B issues. One doc told me that low D can also be a contributor to memory issues. And there are others. I am on a lot of supplements now because it turned out that I did have B and D issues and, because I have moderate macular degeneration, I am on the areds 2 protocol which includes very high doses of several things.
Then there was the 'not paying enough attention' thing. I have always multi tasked and one doc told me that the mind does not handle this as well as we get older so it is important to cut that out and actually pay real attention to what I am doing in the present moment. I work at this daily.
All in all, I am much better. Whether it is the supplements or focusing or what I do not know, but there are things that will help this. Starting with a check up would be a good thing. Meditation helps me because it teaches me to focus and let go of all of the many things that are flying around my brain at any given time, the juggling of which adds to memory impairment. Particularly if you are young, try to get to the bottom of this now before it gets really bad. These things do not remedy themselves. We have to pay attention and take action. I wish you luck with this.