No, please do not do it - not out of date coconut milk or even cow's milk - it is a gardening myth and will only lead to you polluting your houseplant substrates and thereby your houseplants.
Any carton of non--cow's milk will contain preservatives and it is obviously stated there is Salt (Sodium Chloride) as part of the ingredients. Sodium is toxic in even small quantities to houseplants.
You say you do not have any fertiliser. Well, you can buy even the smallest tub of something like water soluble MiracleGro for a few $/£/euros and it will last a very long time indeed.
I have in excess of 100 houseplants and 18 months ago I bought a 2kg tub of MiracleGro fertiliser in a plastic tub off Amazon for about 14 UK pounds. Inside the tub was four x 500g bags and you put 1x15g of powder in a 5 litre watering can of tap water.
So far, I have only just opened the second bag, so I have effectively in 18 months only used 1/4 of the pack. I worked out a few weeks ago it was costing me about 13 cents/pence a month to fertilise all my houseplants as well as some containers outside.
So please, tip the coconut milk down the sink, avoid using cow's milk too and buy yourself some water soluble powder fertiliser which by law HAS to state what is in it and having done a lot of research into what is available, MiracleGro has everything except calcium and magnesium and the drinking/tap water has enough calcium and magnesium to make up the difference.
All the home-remedy type 'fertilisers' are bunkum and have been proven not to work and it is hardly rocket science to work out that buying the proper stuff is the right course of action and also cheaper than messing about with old wives tales.
I may have misread that the plants you want to 'fertilise' were houseplants but the same applies to outdoor plants in the garden.
Oh and if you are talking about outdoor plants, pouring coconut or cow's milk will only attract vermin and other pests who might even dig up your plants.
On a separate subject, if you live in the Global North, it is October at the time of writing and so most plants are shedding leaves or dying and this is not the time to put even commercial fertiliser down on your garden beds.
The best thing you can do for your garden would be to buy a 1 tonne builder's bag of wood mulch and lay a 2-4 ins layer on all your beds. It will slowly break down over the next 2 years to release gradual amounts of root-ready nutrients, will increase the diversity in the soil, adding organic material to enrich it, will keep the ground marginally warmer, acting like a blanket and will keep weeds down through the autumn, winter, spring and summer.
I am getting such a bag delivered tomorrow and in the UK it is costing me £45 (so about $55). I put a very thick blanket of it down on my raised beds last October and there is still plenty there and I had a good crop of raspberries this last summer which I put down to the mulch.
If you live in the USA, I understand that arborists routinely will give wood mulch away for free since it is the shreddings of small branches and hedges that are generally trimmed the year round and it costs them to dispose of it if they do not have premises so getting it for free is a great bonus.
I trust this is a better course of action you can take for your garden beds outdoors?