Imperial County is usually made fun of on Datalounge. But I've been to many of the sights there and I'll offer my 2 cents worth.
I don't think it's worth going all the way to the edge of the Salton Sea. There are other lakes in California that are far more rewarding. For bird watchers, the Sonny Bono Salton Sea National Wildlife Refuge is a reason to go to there. When I was nearby it, an enormous flock of what had to be 50,000 birds passed overhead and that was an astounding sight.
The area southeast of the sea is an active geothermal area, and there are at least 2 geothermal power plants, maybe more. Nearby one of them is the Calipatria Mud Volcanoes, officially called the Davis-Schrimpf Seep Field. They are gurgling, burping hills of mud like something you might see at Yellowstone. There are signs that you are not allowed, but nobody anywhere around to enforce it, so effectively you can visit it. Put plastic bags on your shoes to keep the fine porcelain-like mud off them.
Also due to the geothermal activity there are several hot springs, some free. Slab City Hot Spring, Five Palms Oasis, and High Line Hot Well (often called Holtville Hot Spring). I've been to High Line and it is excellent, free and well mainained in the winter by snowbirds. Ot can get pretty crowded. It the summer it is not as crowded, due to the hot weather, and as well maintained, but it can be very cruisy on a warm summer night. Or really any night. But BLM cops sometimes visit. I've also heard Five Palms hot spring is cruisy, but I haven't been there as you have to go on dirt roads to get to it.
Near the Salton Sea is the famous Slab City, popularized by the film "Into the Wild." The East Jesus art area is near that. Salvation Mountain is nearby, though the man who made it has passed away and I don't know if they are still maintaining it. The International Banana Museum is in Mecca, to the north.
I'll also add that some of the best Mexican food I've ever had has been in El Centro and Imperial County. Fresh produce is also high quality there, due to the many farms. Holtville is the "carrot capital" and they have a carrot festival in February. People who live in this county are generally down to earth and friendly. Sadly there is a lot of poverty and people who are just barely getting by, amidst all the agricultural prosperity.
Also in the general area are the Colorado River, Glamis Dunes off-road recreation area, the vast Anza-Borrego Desert State Park, Agua Caliente Hot Springs County Park, the Desert View Tower, the town of Borrego Springs, with Ricardo Breceda's metal sculptures, Laguna Mountain Recreation Area, still forested with sweeping desert views, Cuyamaca Rancho State Park, Lake Cuyamaca, and the rugged San Jacinto and Santa Rosa Mountains National Monument (thanks Bill Clinton).
Idyllwild is not too far away, you get to it from Palm Desert by taking the Palms-to-Pines Highway (74) up the mountain. Idyllwild is a small mountain town, forested, artsy and gay-friendly. It has hiking trails and rock climbing. Some southern California rock climbers practice there before going on to Yosemite. The best place to eat in Idyllwild is the Gastrognome. More expensive, but worth it.