Peel cooled baked sweet potatoes and place in a large bowl. Place quartered red potatoes in a steamer over 1 inch of boiling water and steam for 15 to 20 minutes, until tender. Remove from heat and add to bowl with sweet potatoes. Mash potatoes together. The red skins will fall apart. Stir in cilantro. Combine 2 tablespoons of the oil with ground red chiles or red pepper flakes, coriander seeds and cumin seeds in a large, heavy skillet and heat over medium-high heat until cumin is fragrant and golden-brown, 1 to 2 minutes. Stir in cayenne and then scrape in potato mixture and salt to taste. Stir in peas and cook just until potatoes and peas are warmed through, about 2 minutes, stirring and scraping the bottom of the skillet often to work in any browned bits. Transfer mixture to a large bowl. Stir in lemon juice, taste and adjust salt, and set aside to cool completely. Take up about 1/3 cup of the mixture and form into a ball (you can wet your hands to reduce sticking). Roll ball in the panko or chickpea flour, then gently flatten into a patty. Set on a plate and continue with the rest of the mixture. Refrigerate uncovered for 1 hour or longer (the longer the better). When you’re ready to cook, place a rack over a sheet pan. Heat 2 tablespoons of oil in a 12-inch, heavy nonstick frying pan over high heat. Swirl pan to coat with the hot oil. Lower heat to medium. Place 4 to 5 patties in the pan (do not crowd), and cook until well browned on one side, about 4 minutes. Turn and brown for about 4 more minutes. Remove to rack. Heat remaining oil in the pan and cook remaining patties. Keep patties warm in a low oven until ready to serve. Serve with a salad and your choice of toppings, such as the usual (ketchup, mustard, relish), or yogurt raita, garlic yogurt, or chutney.