First of all, E does not mean an emulsifier. E is the standardized designation for food additives. Food, mind you. That is, if a supplement has been standardized as E, it is likely relatively safe to eat (a question of dose, but that's another question). In general, everything that is not used directly in food, but can be added to other dishes to improve their characteristics (taste, shelf life, appearance) - everything has an E code, regardless of origin. Citric acid, for example, is found in many fruits and is called E330. In cooking, it is used as an acidifier (you can use vinegar instead) and an antioxidant.
Secondly, emulsifiers themselves are also not harmful. An emulsifier is a substance that makes it easier to mix two immiscible liquids such as water and oil. No matter how much you beat them in one container, the oil will always separate and float. But it's worth adding egg yolk, which has emulsifier properties, then a little vinegar, salt, pepper, mustard and bam - we got mayonnaise. There are often thickeners next to emulsifiers that make the sauces thicker and more convenient to use - it would be difficult to spread ketchup on something if it had the consistency of tomato juice. The most common thickener is starch. Do you eat potatoes or pastries? There are tons of this starch.
To summarize - not everything is so scary and not everything is so bad. Some substances can be dangerous in high concentrations, but most often they are so high that you will not eat so much in a lifetime. Nobody adds frankly dangerous and poisonous additives to food - who can benefit from this?