I would think the majority of the crowds are going to be on the Pacific side, just because there's always more people in and around Ensenada, and because it's the race start/finish area. We've always stayed on the Sea of Cortez side, which isn't too bad. It seems like quite a few folks go home right after the race and I would think that's true for both sides, especially since kids should still be in school. Other than crossing the border back into the US at the San Ysidro crossing on a Sunday (sometimes takes up to 3 hours or more), its a doable "weekend trip" for anyone living in the southern California area. I think the farther away people come from, the more likely they are to stay for a few days and make it worth the drive.
I don't know how long you're planning to stay in Baja, but I would consider crossing the border back into the US during the week. I would also consider one of the other crossings, as the San Ysidro crossing is by far the busiest. You would be surprised at just how many people living in Mexico cross the border into the US in the morning, and especially on Monday morning, to go to work. This makes coming back during the "morning commute" times a bit crowded as well. One last thing about crossing the borders you should be aware of is that not all crossings are open 24 hours. We use the Mexicali East crossing which is only open from 6:00 am to 10:00 pm. If I had to cross at San Ysidro, I would try to do it late at night or super early in the morning. That being said, I generally don't recommend driving in Baja at night if you can avoid it.
As far as camping and safety goes, it's been quite a few years since we've camped on the Pacific side. When we did, we never had any problems, but things are a bit different now compared to 15 years ago. Honestly, I would probably tend to stay in an actual campground, around other people, if I was going to be staying on the Pacific side anywhere from Ensenada to the border.
As far as dispersed camping goes, something you might consider is finding a place along the race route, out away from the crowds. There are literally hundreds of miles of wide open desert for you to explore. In my opinion, one of the coolest things to do is to go out along the race route and find yourself a small hilltop to set up camp and watch the race from, especially if you can find one where the race course goes right along the base of the hill. It gets you up out of the dust a bit and it's pretty awesome to watch them coming from a distance and pass right by you. Make sure to bring a good pair of binoculars if you have them. If you're lucky, you might find one of the many hilltops where somebody has already leveled/cleared a spot and built a fire ring.
As far as dispersed camping on the beach along the Sea of Cortez side, the spot at the north end of the town of San Felipe posted by Jroettele in the thread "10 day trip to Baja" is a good one. After that, you have to get south of town a bit to find a spot where it will be just you. There are a number of "campos" running along the beach from San Felipe to Puerticitos, so you have to try to find a spot somewhere in between them if you want to camp alone. The best thing to do, if you have the time to do a little exploring, is just start driving down some of the dirt roads leading to the beach from Mex. Hwy. 5 until you find an acceptable spot. Other than possibly getting stuck, I think the worst that might happen is the property owner will show up and tell you to leave. However, a much more likely scenario is no one will show up or the property owner will want to charge you to camp on their land. If there aren't signs saying to stay out, fences, gates, Etc., I personally would have no problem giving it a try. If it were me, I would start looking around in the area between Punta Estrella and Rancho Percebu, as this is far enough away from town to avoid the crowds, but close enough to make the trek into town for supplies and such.
As far as safety, I think no matter where you go in Baja, just remember that even though you drove there, you are still in a "foreign country" and you are now playing by their rules. A little respect for them and their culture along with a little common sense goes a long way towards you having a safe trip. There are going to be military check points, some stationary and some mobile. Occasionally the Federal Police set up check points. They might make you get out of your car and they might do a cursory search of your vehicle. They might ask for your driver's license or other documentation like a Tourist Permit. They are going to have rifles hanging off their shoulders and some won't look old enough to be allowed to carry a rifle. All are reminders that you are in a foreign country. Be polite, do what they say, and as long as you are following their rules, you won't have any trouble.
Not to load you up with too much to do, but another side trip you might consider is going to Mike's Sky Ranch (31.1096, -115.6358), about a 2 hour drive from Ensenada. It's a pretty cool place up in the mountains (around 3,800' elevation) with pine trees and even a trout stream. Not usually things you would associate with Baja.
It's a popular spot for dirtbike riders and day trippers. Mex. Hwy. 3 runs roughly east/west between Mex. Hwy. 1 on the Pacific side and Mex. Hwy. 5 on the Sea of Cortez side. This is the road you would take if you were going to San Felipe from Ensenada and it's a little over 100 miles long. Coming from Ensenada it's about 85 miles to the turn off to Mike's.
From there it's about 20 miles of fairly well maintained dirt road to the Ranch. They have quite a few rooms, a swimming pool and a restaurant. Because it's popular with the dirtbike crowd, be prepared to be startled as they come flying around you or towards you on a blind curve.
You might have some issues getting to Mike's on race day and it might be a bit crowded as well. Before and after race day you should be good.
We're going down to our place for a few days sometime in May. If I have time, I'll do a little looking around for a good place for you to camp on the beach. I'll also grab the GPS coordinates for the fossil beds across from Rancho Percebu.