Place a small bowl or ramakin into a large bowl and cut the kernels off of the sweet corn ears using a knife.
Heat a skillet on medium to medium-high heat. Add the 2 teaspoons of olive oil and the corn and grill, stirring occasionally, for about 4 to 5 minutes until starting to brown. Remove from the heat and allow to cool for at least 10 minutes.
Mix the milk and egg together in the same bowl you used to strip the ears. Then stir in the remaining ingredients to form a stiff batter.
Once the corn has cooled slightly, start slowly adding the corn to allow the egg to temper and not cook.
Turn oven on to 170°.
Wipe out the skillet and add vegetable oil to cover the bottom by 1/4 inch, keeping on medium heat. Heat oil to 350°.
Use a #24 scoop or 1/4 cup measure to add the corn batter to the hot oil, being careful not to splatter oil. Flatten the batter with a small metal spatula or the bottom of the 1/4 measuring cup. Fry 3 or 4 fritters at a time but avoid over crowding the skillet.
Fry for 2 to 2 1/2 minutes until browned, turn and fry for another 2 to 2 1/2 minutes. Turn the heat down if the fritters are spitting and browning too rapidly.
Place patties on a paper towel-lined sheet pan to soak up excess oil and place in the oven to keep warm. Continue to fry the remaining batter.
Serve warm with a dipping sauce or dressing.