Releasing a Decade of Forex Tick Data I Crawled and Converted
In my exploration of the world of big data and I became curious about tick data. Unfortunately, market data is almost always behind a paywall or de-sampled to the point of uselessness. After discovering the Dukascopy API, I knew I wanted to make this data available for all in a more accessible format. Over the course of a few months, I downloaded, cleaned, parsed, and compressed over a decade of Forex tick data on 37 currency pairs and commodities. Today I am happy to finally release the final result of my work to the DataHoarder community!
The data was collected from https://www.dukascopy.com/ via a public API that allows for the download of tick data on the hour level. These files come in the form of a .bi5 file. The data starts as early as 2004 all the way to 2019. These files were decompressed, then merged into yearly CSV’s named in the following convention. “AUDCHF_tick_UTC+0_00_2011.csv” or ‘Pair_Resolution_Timezone_Year.csv’ These CSV’s are split into 3 categories “Majors”, “Crosses”, “Commodities”. Majors, Crosses, and Commodities have had their timestamps modified so that they are in the official UTC ISO standard. This was originally done for a Postgresql database that quickly became obsolesced. Any files that have been modified are appended with a “-Parse”. These timestamps have been modified in the following format. Millisecond timestamps to UTC +00:00 time [2017.01.01 22:37:08.014] -- [2017-01-01T22:37:08.014+00:00] https://preview.redd.it/x6g277skfiu51.png?width=1399&format=png&auto=webp&s=35cd6735c1826424580919ac3377612377a3107c
For those looking to use this data in a live context or update it frequently, I have included a number of tools for both Windows and Linux that will be useful.
The ~/dukascopy/resources/windows folder contains a third party tool written in java that can download and convert Dukascopy’s .bi5 files. I have also included the latest zstd binaries from Zstandard Github page.
Linux is my daily driver in 99% of cases, so I have developed all my scraping tools using Linux only tools. In the ~/dukascopy/resources/linux folder you will find a number of shell script and pyhton3 files that I used to collect this data. There are quite a few files in this directory but I will cover the core ones below.
This file is used to download a single symbol for a single day and then convert and merge all 24 .bi5 files into a single CSV.
This file is used to download a single symbol for a full year and then convert and merge all .bi5 files into a single CSV.
This file contains all the core logic for downloading and converting data from dukascopy.
This tad slow but works well enough. It requires the pandas project and parses timestamps into the UTC ISO standard. This is useful for those looking to maintain the format of new files with the those in this repo, or those looking to use this in a SQL database.
No Agent Taobao Direct Buying Guide! Let's view all baby and determine
Taobao Direct Guide for users familiar with 3rd party agents and navigating taobao (with chrome google translate on, hence the title)
Taobao direct consolidation and shipping is available in the following countries: USA, Canada, Australia, New Zealand, Japan, Malaysia, Singapore, Hong Kong, Macau, Taiwan. This review is primarily geared towards the US as that’s where I live.
What is Taobao direct? Basically instead of copying and pasting the item URL into the agent website, you add items to your cart like a regular ecommerce site, check out, wait for items to arrive in the warehouse (similar to what happens when you use an agent) and then when all your items from various sellers are in, you request the logistics company to send everything to you. Disclaimer: I have no Chinese fluency written or otherwise. I did everything through Google translate and my experience with how tb works through agents. If something goes wrong I will probably write off the item 🤣 if you communicate a lot with the ts who use translators it also helps get your point across. If you type in English in tb live chat they will redirect you to the HK/tw help staff who have medium English. Also I bought items I purchased previously with an agent or vouched for here on RL or had crazy high reviews/ratings. Pros:
Duck the agent fees and exchange rate bs
Shipping is cheaper but EMS is the only option right now
If you are overcharged for shipping you can get a refund
It’s also easier to collect the discounts and coupons from stores (ymmv with agent)
Not good for reps or mainstream branded rep items as they can be marked as contraband by taobao and then you have to get a refund and return to the seller
You pay 5% sales tax (see note at the end) and 3% on the total payment through alipay with a foreign cc (regardless if your cc does not charge forex)
No qc pics and can't really control the declared value if that matters
Other categories of items like make up brushes, liquids, powders, sharps, etc. are not eligible
20 day hold limit apparently every day after that is charged 1 yuan per item per day
I think the ideal usage for taobao direct would be light items like innerwear, jewelry, soft/non fragile goods, generally clothing and shoes although I don’t know if they will include the box by default. Please see here for the image guide for ordering Sorry in advance if my descriptions are wonky, I'm not great at following OR writing instructions but hopefully the screenshots make it easier to follow along.
Create an account (there are various guides out there for overseas members) and go into your account and add your home address (or the superbuy warehouse address)
Find your items and change the delivery location to "overseas", add to cart
When you're ready to check out hit check out, enter your cc info on the alipay (remember to use a card that doesn't charge foreign transaction fees) and confirm it goes through.
Wait for all your stuff to come in. When its in the tb warehouse it will show up in the "consolidated delivery" section tagged with a weight (usually volumetric or actual). The 20 day countdown will start once its available for international shipping.
After all your items are in, or you can batch up by selecting items on the consolidated delivery page, submit for delivery. Pay again through alipay.
Use the check logistics option to get the tracking info and wait for your haul!
After receiving but before you open, take photos of it on a scale and the lxwxh with a ruler as well. This is because they will overestimate your shipping but there isn't rehearsal shipping like with agents. You can request a refund after the fact with the "refund/complaint" option on the consolidated delivery page (mine says check refund because I've already gone through it)
Getting a refund: select the "only refund" option, "goods received" and "shipping cost does not match" and leave the full shipping amount in. Upload your measurement and weight photos (make sure the file size is not too big). Within 72hr they will reply and ask you to modify your application with the real amount owed (if any). It will go back to your cc through alipay (may take a few days).
Cost comparison: Even after the 5% sales tax and 3% alipay, it cost me $6.20 total from my credit card statement. A 39 yuan top up for sb is $6.53 as of today (if using paypal). For some the qc pictures and the longer storage period are well worth the difference. However a good compromise is the parcel forwarding option in sb. Instead of shipping to your house you can set up superbuy’s warehouse address and pay in taobao and wait for your items to show up in sb. You also have to submit the item link and the tracking # in superbuy so they can find your stuff. There's no sales tax and usually no shipping and you can select the coupons you want. I had a pair of pants make it to the sb warehouse almost 24hr after ordering, and another 24hr after entering my shipping info and item link in sb, it showed up in my account with free (non hd) pictures of the item. Then I cried putting together the shipping parcel lol. This is a good way to dodge the sales tax and hold items for longer. However then you're at the mercy of the shipping costs (but you do have more options for delivery lines and you can customize how you want your items packaged too). The taobao warehouse will really throw everything in there, probably in a poly envelope. The taobao shipping rates are 90yuan for the first .5kg and 48 yuan per every .5 after which is very competitive even after accounting for volumetric weight. Sb ems starts at 186 for the first .5kg and 61y every .5kg after. Of course rates and terms are subject to change with the times. I had a package that came in at 277g when I measured it at home but I was charged for 1.6kg. After sending in the package images they refunded 144yuan (the true volumetric weight was about .97kg.) Taobao volumetric calculation is lxwxh (cm)/6000. Timeline wise I submitted 8/16 and received 8/28 although I think because it was so light they used epacket/china post because it was not an EMS tracking # big sigh. Still less than 10 days can't complain. Hope this helps! I'm sure I missed something on this guide so feel free to leave any questions and I will update the post accordingly. Apologies this is very us-centric, I also cannot comment on getting a refund or exchange from sellers before you ship out but there is now english support (albeit a bit wonky) through chat and aliwangwang+google translate can get you pretty far. Ps: highly recommend using the app too as its easier to get chat messages from the seller. You can screenshot and upload images to Google translate to read the text.
I'll leave these as they are for a little while (About 12 hours from now). If you check the trade history of these accounts vrs my posting history you'll see none of the losses made were from trades I posted. Actually sometimes I inverted the trades I posted to generate losses. You can see this in the "History" tab and of course you know how to see my Reddit posts. Heading into Monday I will attach new accounts to this and we can then run phase two of the test. I could just upload accounts already running and proftable to this, but I think it's best to do everything in real time and in the open. From now on I will be trading the same positions as I post and tracking the results of these. You can find these results at the links above as of Monday.
Now I'll trade and track results properly and we get to do a proper experiement into the real motives and nature of everyone involved. I've said it for months, the truth of people will be revealed by their own actions. Just a case of waiting untl the time is right. From now on I'll only trade the things I post in my subs/forum. No messing about. If they bomb this time, I suck. If not, let's see what happens :)
Hello, I'm a new trader. I'm 23 years. I just started trading forex on MT4, my broker is VantageFx. I made like 1k5€ in 3 days ( this was the first time I traded ). But it's still not real for me, because it was really easy to make this money with no experience at all. So I'm wondering if it's a trap or is it real ? I searched all over internet and everytime they say it's really hard to live from trading, and I see a lot of people loosing a lot of money but I don't know how did they do it. But I don't know if I was lucky and made 1k5€ just from pure luck or is trading forex is really easy ? Sorry for my Bad english, I'm french. Edit : Thank you for all your feed back ! I will cash out the money I originaly put and I will keep testing my strat. I will make another post in maybe 1 month to update my mindstate and share with you my conclusion.
Forex Signals Reddit: top providers review (part 1)
Forex Signals - TOP Best Services. Checked!
To invest in the financial markets, we must acquire good tools that help us carry out our operations in the best possible way. In this sense, we always talk about the importance of brokers, however, signal systems must also be taken into account. The platforms that offer signals to invest in forex provide us with alerts that will help us in a significant way to be able to carry out successful operations. For this reason, we are going to tell you about the importance of these alerts in relation to the trading we carry out, because, without a doubt, this type of system will provide us with very good information to invest at the right time and in the best assets in the different markets. financial Within this context, we will focus on Forex signals, since it is the most important market in the world, since in it, multiple transactions are carried out on a daily basis, hence the importance of having an alert system that offers us all the necessary data to invest in currencies. Also, as we all already know, cryptocurrencies have become a very popular alternative to investing in traditional currencies. Therefore, some trading services/tools have emerged that help us to carry out successful operations in this particular market. In the following points, we will detail everything you need to know to start operating in the financial markets using trading signals: what are signals, how do they work, because they are a very powerful help, etc. Let's go there!
What are Forex Trading Signals?
https://preview.redd.it/vjdnt1qrpny51.jpg?width=640&format=pjpg&auto=webp&s=bc541fc996701e5b4dd940abed610b59456a5625 Before explaining the importance of Forex signals, let's start by making a small note so that we know what exactly these alerts are. Thus, we will know that the signals on the currency market are received by traders to know all the information that concerns Forex, both for assets and for the market itself. These alerts allow us to know the movements that occur in the Forex market and the changes that occur in the different currency pairs. But the great advantage that this type of system gives us is that they provide us with the necessary information, to know when is the right time to carry out our investments.
In other words, through these signals, we will know the opportunities that are presented in the market and we will be able to carry out operations that can become quite profitable.
Profitability is precisely another of the fundamental aspects that must be taken into account when we talk about Forex signals since the vast majority of these alerts offer fairly reliable data on assets. Similarly, these signals can also provide us with recommendations or advice to make our operations more successful.
»Purpose: predict movements to carry out Profitable Operations
In short, Forex signal systems aim to predict the behavior that the different assets that are in the market will present and this is achieved thanks to new technologies, the creation of specialized software, and of course, the work of financial experts. In addition, it must also be borne in mind that the reliability of these alerts largely lies in the fact that they are prepared by financial professionals. So they turn out to be a perfect tool so that our investments can bring us a greater number of benefits.
The best signal services today
We are going to tell you about the 3 main alert system services that we currently have on the market. There are many more, but I can assure these are not scams and are reliable. Of course, not 100% of trades will be a winner, so please make sure you apply proper money management and risk management system.
1. 1000pipbuilder (top choice)
Fast track your success and follow the high-performance Forex signals from 1000pip Builder. These Forex signals are rated 5 stars on Investing.com, so you can follow every signal with confidence. All signals are sent by a professional trader with over 10 years investment experience. This is a unique opportunity to see with your own eyes how a professional Forex trader trades the markets. The 1000pip Builder Membership is ordinarily a signal service for Forex trading. You will get all the facts you need to successfully comply with the trading signals, set your stop loss and take earnings as well as additional techniques and techniques! You will get easy to use trading indicators for Forex Trades, including your entry, stop loss and take profit. Overall, the earnings target per months is 350 Pips, depending on your funding this can be a high profit per month! (In fact, there is by no means a guarantee, but the past months had been all between 600 – 1000 Pips). >>>Know more about 1000pipbuilder Your 1000pip builder membership gives you all in hand you want to start trading Forex with success. Read the directions and wait for the first signals. You can trade them inside your demo account first, so you can take a look at the performance before you make investments real money! Features:
Forex signals sent by email and SMS
Entry price, take profit and stop loss provided
Suitable for all time zones (signals sent over 24 hours)
Digital Derivatives Markets (DDMarkets) have been providing trade alert offerings since May 2014 - fully documenting their change ideas in an open and transparent manner. September 2020 performance report for DD Markets. Their manner is simple: carry out extensive research, share their evaluation and then deliver a trading sign when triggered. Once issued, daily updates on the trade are despatched to members via email. It's essential to note that DDMarkets do not tolerate floating in an open drawdown in an effort to earnings at any cost - a common method used by less professional providers to 'fudge' performance statistics. Verified Statistics: Not independently verified. Price: plans from $74.40 per month. Year Founded: 2014 Suitable for Beginners: Yes, (includes handy to follow trade analysis) VISIT -------
If you are looking or a forex signal service with a reliable (and profitable) music record you can't go previous Joel Kruger and the team at JKonFX. Trading performance file for JKonFX. Joel has delivered a reputable +59.18% journal performance for 2016, imparting real-time technical and fundamental insights, in an extremely obvious manner, to their 30,000+ subscriber base. Considered a low-frequency trader, alerts are only a small phase of the overall JKonFX subscription. If you're searching for hundreds of signals, you may want to consider other options. Verified Statistics: Not independently verified. Price: plans from $30 per month. Year Founded: 2014 Suitable for Beginners: Yes, (includes convenient to follow videos updates). VISIT
The importance of signals to invest in Forex
Once we have known what Forex signals are, we must comment on the importance of these alerts in relation to our operations. As we have already told you in the previous paragraph, having a system of signals to be able to invest is quite advantageous, since, through these alerts, we will obtain quality information so that our operations end up being a true success.
»Use of signals for beginners and experts
In this sense, we have to say that one of the main advantages of Forex signals is that they can be used by both beginners and trading professionals. As many as others can benefit from using a trading signal system because the more information and resources we have in our hands. The greater probability of success we will have. Let's see how beginners and experts can take advantage of alerts:
Beginners: for inexperienced these alerts become even more important since they will thus have an additional tool that will guide them to carry out all operations in the Forex market.
Professionals: In the same way, professionals are also recommended to make use of these alerts, so they have adequate information to continue bringing their investments to fruition.
Now that we know that both beginners and experts can use forex signals to invest, let's see what other advantages they have.
When we dedicate ourselves to working in the financial world, none of us can spend 24 hours in front of the computer waiting to perform the perfect operation, it is impossible. That is why Forex signals are important, because, in order to carry out our investments, all we will have to do is wait for those signals to arrive, be attentive to all the alerts we receive, and thus, operate at the right time according to the opportunities that have arisen. It is fantastic to have a tool like this one that makes our work easier in this regard.
»Carry out profitable Forex operations
These signals are also important, because the vast majority of them are usually quite profitable, for this reason, we must get an alert system that provides us with accurate information so that our operations can bring us great benefits. But in addition, these Forex signals have an added value and that is that they are very easy to understand, therefore, we will have a very useful tool at hand that will not be complicated and will end up being a very beneficial weapon for us.
»Decision support analysis
A system of currency market signals is also very important because it will help us to make our subsequent decisions. We cannot forget that, to carry out any type of operation in this market, previously, we must meditate well and know the exact moment when we will know that our investments are going to bring us profits . Therefore, all the information provided by these alerts will be a fantastic basis for future operations that we are going to carry out.
»Trading Signals made by professionals
Finally, we have to recall the idea that these signals are made by the best professionals. Financial experts who know perfectly how to analyze the movements that occur in the market and changes in prices. Hence the importance of alerts, since they are very reliable and are presented as a necessary tool to operate in Forex and that our operations are as profitable as possible.
What should a signal provider be like?
https://preview.redd.it/j0ne51jypny51.png?width=640&format=png&auto=webp&s=5578ff4c42bd63d5b6950fc6401a5be94b97aa7f As you have seen, Forex signal systems are really important for our operations to bring us many benefits. For this reason, at present, there are multiple platforms that offer us these financial services so that investing in currencies is very simple and fast. Before telling you about the main services that we currently have available in the market, it is recommended that you know what are the main characteristics that a good signal provider should have, so that, at the time of your choice, you are clear that you have selected one of the best systems.
»Must send us information on the main currency pairs
In this sense, one of the first things we have to comment on is that a good signal provider, at a minimum, must send us alerts that offer us information about the 6 main currencies, in this case, we refer to the euro, dollar, The pound, the yen, the Swiss franc, and the Canadian dollar. Of course, the data you provide us will be related to the pairs that make up all these currencies. Although we can also find systems that offer us information about other minorities, but as we have said, at a minimum, we must know these 6.
»Trading tools to operate better
Likewise, signal providers must also provide us with a large number of tools so that we can learn more about the Forex market.
We refer, for example, to technical analysis above all, which will help us to develop our own strategies to be able to operate in this market.
These analyzes are always prepared by professionals and study, mainly, the assets that we have available to invest.
»Different Forex signals reception channels
They must also make available to us different ways through which they will send us the Forex signals, the usual thing is that we can acquire them through the platform's website, or by a text message and even through our email. In addition, it is recommended that the signal system we choose sends us a large number of alerts throughout the day, in order to have a wide range of possibilities.
»Free account and customer service
Other aspects that we must take into account to choose a good signal provider is whether we have the option of receiving, for a limited time, alerts for free or the profitability of the signals they emit to us. Similarly, a final aspect that we must emphasize is that a good signal system must also have excellent customer service, which is available to us 24 hours a day and that we can contact them at through an email, a phone number, or a live chat, for greater immediacy. Well, having said all this, in our last section we are going to tell you which are the best services currently on the market. That is, the most suitable Forex signal platforms to be able to work with them and carry out good operations. In this case, we will talk about ForexPro Signals, 365 Signals and Binary Signals.
Forex Signals Reddit: conclusion
To be able to invest properly in the Forex market, it is convenient that we get a signal system that provides us with all the necessary information about this market. It must be remembered that Forex is a very volatile market and therefore, many movements tend to occur quickly. Asset prices can change in a matter of seconds, hence the importance of having a system that helps us analyze the market and thus know, what is the right time for us to start operating. Therefore, although there are currently many signal systems that can offer us good services, the three that we have mentioned above are the ones that are best valued by users, which is why they are the best signal providers that we can choose to carry out. our investments. Most of these alerts are quite profitable and in addition, these systems usually emit a large number of signals per day with full guarantees. For all this, SignalsForexPro, Signals365, or SignalsBinary are presented as fundamental tools so that we can obtain a greater number of benefits when we carry out our operations in the currency market.
I’ve been looking for a broker that has an API for index futures and ideally also futures options. I’m looking to use the API to build a customized view of my risk based on balances, positions, and market conditions. Searching the algotrading sub I found many API-related posts, but then when I actually read them and their comments, I found they’re often lacking in real substance. It turns out many brokers or data services that have APIs don’t actually support index futures and options via the API, and instead they focus on equities, forex, or cypto. So here’s the list of what I’ve found so far. This isn’t a review of these brokers or APIs and note that I have a specific application in mind (index futures and futures options). Perhaps you’re looking for an API for equities, or you just want data and not a broker, in which case there may be a few options. Also, I’m based in the US so I didn’t really look for brokers or platforms outside the US. If you have experience with these APIs, please chime in with your thoughts. Also, I may have missed some brokers or platforms. If I did or if you see anything that needs correction please let me know.
Broker with a variety of platforms including CQG, Rithmic, TT, some with APIs
Wow, this list grew longer than I originally thought it would be. If you spot a mistake, please let me know and I’ll correct it. Edit: - added Lightspeed API - updated Dashprime to indicate some of the APIs available - added Medved Trader to table - added marketstack to table
I want to tell you quickly the nightmare I'm going through. First, the client service is... Non existent. Let me repeat, no one will help you. I sent approximately 10 emails in a week, I had two automatic replies spewing the same nonsensical pre programed gibberish. The chat doesn't work. I waited hours, no one ever responded to my request. They did once when I had an issue with currencies, and they actually made me open a forex position, while there was a conversion tool... Now with regards to the phone service. You have to wait approximately one hour before someone picks up. They are arrogant, useless and impersonal. Yesterday for example I bounced back and forth between three different people, each having no idea what was my request. The past weeks, after hours and hours on the line, no one could help me or even listen to me without interrupting me. I have been waiting for ten days now to have the options '' granted ''. This broker feels like they are your university teachers that take your money but won't help you even if their lives depended on it. I still cannot use options, bonds, futures. Just stocks. Also, very often I don't receive the text or phone call to log in to the platform. I waited 2h the other day before I could log in. You have to update the page constantly because nothing is loading. Your balance or other information won't appear. The orders appear and disappear without explanation, sometimes I have to place an order 2 or 3 times before it's there. The interface looks like a software from the 90's, unusable and frankly it hurts my eyes. It needs a serious update. In ten days I haven't been able to trade one day. I made a couple thousands with my paper account, and I wish I could have done that with my real money account. I'm switching to Degiro, waiting for the paperwork to be done. You can find similar experiences all over the Internet. Don't give them commissions, I dearly regret using this fraud of a broker.
Whatsup forex! As a few of you know (those who read my initial post anyway) I went live this week with 250AUD. Just thought I'd give a quick update for those who care, if it's not appropriate mods please let me know and I'll be on my merry way. I took 2 trades this week. As per the laws of the universe, my first ever live trade was a bust, annihilated by a full day's ATR worth of reversal. I lost 1.78%. Second trade went better, ended up taking 3.55% and finishing the week up 1.67%. That's about 4AUD. It was largely a boring affair, and I made a small adjustment to my process. Having real money on the table, even if it's only 4 bucks, has actually helped me be much more focused this week which has been cool. Anyway just wanted to share, feel free to ask whatever, I don't think two live trades make me an expert but hey. Cheers.
Disclaimer: None of this is financial advice. I have no idea what I'm doing. Please do your own research or you will certainly lose money. I'm not a statistician, data scientist, well-seasoned trader, or anything else that would qualify me to make statements such as the below with any weight behind them. Take them for the incoherent ramblings that they are. TL;DR at the bottom for those not interested in the details. This is a bit of a novel, sorry about that. It was mostly for getting my own thoughts organized, but if even one person reads the whole thing I will feel incredibly accomplished.
For those of you not familiar, please see the various threads on this trading system here. I can't take credit for this system, all glory goes to ParallaxFX! I wanted to see how effective this system was at H1 for a couple of reasons: 1) My current broker is TD Ameritrade - their Forex minimum is a mini lot, and I don't feel comfortable enough yet with the risk to trade mini lots on the higher timeframes(i.e. wider pip swings) that ParallaxFX's system uses, so I wanted to see if I could scale it down. 2) I'm fairly impatient, so I don't like to wait days and days with my capital tied up just to see if a trade is going to win or lose. This does mean it requires more active attention since you are checking for setups once an hour instead of once a day or every 4-6 hours, but the upside is that you trade more often this way so you end up winning or losing faster and moving onto the next trade. Spread does eat more of the trade this way, but I'll cover this in my data below - it ends up not being a problem. I looked at data from 6/11 to 7/3 on all pairs with a reasonable spread(pairs listed at bottom above the TL;DR). So this represents about 3-4 weeks' worth of trading. I used mark(mid) price charts. Spreadsheet link is below for anyone that's interested.
I'm pretty much using ParallaxFX's system textbook, but since there are a few options in his writeups, I'll include all the discretionary points here:
I'm using the stop entry version - so I wait for the price to trade beyond the confirmation candle(in the direction of my trade) before entering. I don't have any data to support this decision, but I've always preferred this method over retracement-limit entries. Maybe I just like the feeling of a higher winrate even though there can be greater R:R using a limit entry. Variety is the spice of life.
I put my stop loss right at the opposite edge of the confirmation candle. NOT at the edge of the 2-candle pattern that makes up the system. I'll get into this more below - not enough trades are saved to justify the wider stops. (Wider stop means less $ per pip won, assuming you still only risk 1%).
All my profit/loss statistics are based on a 1% risk per trade. Because 1 is real easy to multiply.
There are definitely some questionable trades in here, but I tried to make it as mechanical as possible for evaluation purposes. They do fit the definitions of the system, which is why I included them. You could probably improve the winrate by being more discretionary about your trades by looking at support/resistance or other techniques.
I didn't use MBB much for either entering trades, or as support/resistance indicators. Again, trying to be pretty mechanical here just for data collection purposes. Plus, we all make bad trading decisions now and then, so let's call it even.
As stated in the title, this is for H1 only. These results may very well not play out for other time frames - who knows, it may not even work on H1 starting this Monday. Forex is an unpredictable place.
I collected data to show efficacy of taking profit at three different levels: -61.8%, -100% and -161.8% fib levels described in the system using the passive trade management method(set it and forget it). I'll have more below about moving up stops and taking off portions of a position.
And now for the fun. Results!
Total Trades: 241
TP at -61.8%: 177 out of 241: 73.44%
TP at -100%: 156 out of 241: 64.73%
TP at -161.8%: 121 out of 241: 50.20%
Adjusted Proft % (takes spread into account):
TP at -61.8%: 5.22%
TP at -100%: 23.55%
TP at -161.8%: 29.14%
As you can see, a higher target ended up with higher profit despite a much lower winrate. This is partially just how things work out with profit targets in general, but there's an additional point to consider in our case: the spread. Since we are trading on a lower timeframe, there is less overall price movement and thus the spread takes up a much larger percentage of the trade than it would if you were trading H4, Daily or Weekly charts. You can see exactly how much it accounts for each trade in my spreadsheet if you're interested. TDA does not have the best spreads, so you could probably improve these results with another broker. EDIT: I grabbed typical spreads from other brokers, and turns out while TDA is pretty competitive on majors, their minors/crosses are awful! IG beats them by 20-40% and Oanda beats them 30-60%! Using IG spreads for calculations increased profits considerably (another 5% on top) and Oanda spreads increased profits massively (another 15%!). Definitely going to be considering another broker than TDA for this strategy. Plus that'll allow me to trade micro-lots, so I can be more granular(and thus accurate) with my position sizing and compounding.
A Note on Spread
As you can see in the data, there were scenarios where the spread was 80% of the overall size of the trade(the size of the confirmation candle that you draw your fibonacci retracements over), which would obviously cut heavily into your profits. Removing any trades where the spread is more than 50% of the trade width improved profits slightly without removing many trades, but this is almost certainly just coincidence on a small sample size. Going below 40% and even down to 30% starts to cut out a lot of trades for the less-common pairs, but doesn't actually change overall profits at all(~1% either way). However, digging all the way down to 25% starts to really make some movement. Profit at the -161.8% TP level jumps up to 37.94% if you filter out anything with a spread that is more than 25% of the trade width! And this even keeps the sample size fairly large at 187 total trades. You can get your profits all the way up to 48.43% at the -161.8% TP level if you filter all the way down to only trades where spread is less than 15% of the trade width, however your sample size gets much smaller at that point(108 trades) so I'm not sure I would trust that as being accurate in the long term. Overall based on this data, I'm going to only take trades where the spread is less than 25% of the trade width. This may bias my trades more towards the majors, which would mean a lot more correlated trades as well(more on correlation below), but I think it is a reasonable precaution regardless.
Time of Day
Time of day had an interesting effect on trades. In a totally predictable fashion, a vast majority of setups occurred during the London and New York sessions: 5am-12pm Eastern. However, there was one outlier where there were many setups on the 11PM bar - and the winrate was about the same as the big hours in the London session. No idea why this hour in particular - anyone have any insight? That's smack in the middle of the Tokyo/Sydney overlap, not at the open or close of either. On many of the hour slices I have a feeling I'm just dealing with small number statistics here since I didn't have a lot of data when breaking it down by individual hours. But here it is anyway - for all TP levels, these three things showed up(all in Eastern time):
7pm-4am: Fewer setups, but winrate high.
5am-6am: Lots of setups, but but winrate low.
12pm-3pm Medium number of setups, but winrate low.
I don't have any reason to think these timeframes would maintain this behavior over the long term. They're almost certainly meaningless. EDIT: When you de-dup highly correlated trades, the number of trades in these timeframes really drops, so from this data there is no reason to think these timeframes would be any different than any others in terms of winrate. That being said, these time frames work out for me pretty well because I typically sleep 12am-7am Eastern time. So I automatically avoid the 5am-6am timeframe, and I'm awake for the majority of this system's setups.
Moving stops up to breakeven
This section goes against everything I know and have ever heard about trade management. Please someone find something wrong with my data. I'd love for someone to check my formulas, but I realize that's a pretty insane time commitment to ask of a bunch of strangers. Anyways. What I found was that for these trades moving stops up...basically at all...actually reduced the overall profitability. One of the data points I collected while charting was where the price retraced back to after hitting a certain milestone. i.e. once the price hit the -61.8% profit level, how far back did it retrace before hitting the -100% profit level(if at all)? And same goes for the -100% profit level - how far back did it retrace before hitting the -161.8% profit level(if at all)? Well, some complex excel formulas later and here's what the results appear to be. Emphasis on appears because I honestly don't believe it. I must have done something wrong here, but I've gone over it a hundred times and I can't find anything out of place.
Moving SL up to 0% when the price hits -61.8%, TP at -100%
Adjusted Proft % (takes spread into account): 5.36%
Taking half position off at -61.8%, moving SL up to 0%, TP remaining half at -100%
Adjusted Proft % (takes spread into account): -1.01% (yes, a net loss)
Now, you might think exactly what I did when looking at these numbers: oof, the spread killed us there right? Because even when you move your SL to 0%, you still end up paying the spread, so it's not truly "breakeven". And because we are trading on a lower timeframe, the spread can be pretty hefty right? Well even when I manually modified the data so that the spread wasn't subtracted(i.e. "Breakeven" was truly +/- 0), things don't look a whole lot better, and still way worse than the passive trade management method of leaving your stops in place and letting it run. And that isn't even a realistic scenario because to adjust out the spread you'd have to move your stoploss inside the candle edge by at least the spread amount, meaning it would almost certainly be triggered more often than in the data I collected(which was purely based on the fib levels and mark price). Regardless, here are the numbers for that scenario:
Moving SL up to 0% when the price hits -61.8%, TP at -100%
Winrate(breakeven doesn't count as a win): 46.4%
Adjusted Proft % (takes spread into account): 17.97%
Taking half position off at -61.8%, moving SL up to 0%, TP remaining half at -100%
Winrate(breakeven doesn't count as a win): 65.97%
Adjusted Proft % (takes spread into account): 11.60%
From a literal standpoint, what I see behind this behavior is that 44 of the 69 breakeven trades(65%!) ended up being profitable to -100% after retracing deeply(but not to the original SL level), which greatly helped offset the purely losing trades better than the partial profit taken at -61.8%. And 36 went all the way back to -161.8% after a deep retracement without hitting the original SL. Anyone have any insight into this? Is this a problem with just not enough data? It seems like enough trades that a pattern should emerge, but again I'm no expert. I also briefly looked at moving stops to other lower levels (78.6%, 61.8%, 50%, 38.2%, 23.6%), but that didn't improve things any. No hard data to share as I only took a quick look - and I still might have done something wrong overall. The data is there to infer other strategies if anyone would like to dig in deep(more explanation on the spreadsheet below). I didn't do other combinations because the formulas got pretty complicated and I had already answered all the questions I was looking to answer.
2-Candle vs Confirmation Candle Stops
Another interesting point is that the original system has the SL level(for stop entries) just at the outer edge of the 2-candle pattern that makes up the system. Out of pure laziness, I set up my stops just based on the confirmation candle. And as it turns out, that is much a much better way to go about it. Of the 60 purely losing trades, only 9 of them(15%) would go on to be winners with stops on the 2-candle formation. Certainly not enough to justify the extra loss and/or reduced profits you are exposing yourself to in every single other trade by setting a wider SL. Oddly, in every single scenario where the wider stop did save the trade, it ended up going all the way to the -161.8% profit level. Still, not nearly worth it.
As I've said many times now, I'm really not qualified to be doing an analysis like this. This section in particular. Looking at shared currency among the pairs traded, 74 of the trades are correlated. Quite a large group, but it makes sense considering the sort of moves we're looking for with this system. This means you are opening yourself up to more risk if you were to trade on every signal since you are technically trading with the same underlying sentiment on each different pair. For example, GBP/USD and AUD/USD moving together almost certainly means it's due to USD moving both pairs, rather than GBP and AUD both moving the same size and direction coincidentally at the same time. So if you were to trade both signals, you would very likely win or lose both trades - meaning you are actually risking double what you'd normally risk(unless you halve both positions which can be a good option, and is discussed in ParallaxFX's posts and in various other places that go over pair correlation. I won't go into detail about those strategies here). Interestingly though, 17 of those apparently correlated trades ended up with different wins/losses. Also, looking only at trades that were correlated, winrate is 83%/70%/55% (for the three TP levels). Does this give some indication that the same signal on multiple pairs means the signal is stronger? That there's some strong underlying sentiment driving it? Or is it just a matter of too small a sample size? The winrate isn't really much higher than the overall winrates, so that makes me doubt it is statistically significant. One more funny tidbit: EUCAD netted the lowest overall winrate: 30% to even the -61.8% TP level on 10 trades. Seems like that is just a coincidence and not enough data, but dang that's a sucky losing streak. EDIT: WOW I spent some time removing correlated trades manually and it changed the results quite a bit. Some thoughts on this below the results. These numbers also include the other "What I will trade" filters. I added a new worksheet to my data to show what I ended up picking.
Total Trades: 75
TP at -61.8%: 84.00%
TP at -100%: 73.33%
TP at -161.8%: 60.00%
Moving SL up to 0% when the price hits -61.8%, TP at -100%: 53.33%
Taking half position off at -61.8%, moving SL up to 0%, TP remaining half at -100%: 53.33% (yes, oddly the exact same winrate. but different trades/profits)
Adjusted Proft % (takes spread into account):
TP at -61.8%: 18.13%
TP at -100%: 26.20%
TP at -161.8%: 34.01%
Moving SL up to 0% when the price hits -61.8%, TP at -100%: 19.20%
Taking half position off at -61.8%, moving SL up to 0%, TP remaining half at -100%: 17.29%
To do this, I removed correlated trades - typically by choosing those whose spread had a lower % of the trade width since that's objective and something I can see ahead of time. Obviously I'd like to only keep the winning trades, but I won't know that during the trade. This did reduce the overall sample size down to a level that I wouldn't otherwise consider to be big enough, but since the results are generally consistent with the overall dataset, I'm not going to worry about it too much. I may also use more discretionary methods(support/resistance, quality of indecision/confirmation candles, news/sentiment for the pairs involved, etc) to filter out correlated trades in the future. But as I've said before I'm going for a pretty mechanical system. This brought the 3 TP levels and even the breakeven strategies much closer together in overall profit. It muted the profit from the high R:R strategies and boosted the profit from the low R:R strategies. This tells me pair correlation was skewing my data quite a bit, so I'm glad I dug in a little deeper. Fortunately my original conclusion to use the -161.8 TP level with static stops is still the winner by a good bit, so it doesn't end up changing my actions. There were a few times where MANY (6-8) correlated pairs all came up at the same time, so it'd be a crapshoot to an extent. And the data showed this - often then won/lost together, but sometimes they did not. As an arbitrary rule, the more correlations, the more trades I did end up taking(and thus risking). For example if there were 3-5 correlations, I might take the 2 "best" trades given my criteria above. 5+ setups and I might take the best 3 trades, even if the pairs are somewhat correlated. I have no true data to back this up, but to illustrate using one example: if AUD/JPY, AUD/USD, CAD/JPY, USD/CAD all set up at the same time (as they did, along with a few other pairs on 6/19/20 9:00 AM), can you really say that those are all the same underlying movement? There are correlations between the different correlations, and trying to filter for that seems rough. Although maybe this is a known thing, I'm still pretty green to Forex - someone please enlighten me if so! I might have to look into this more statistically, but it would be pretty complex to analyze quantitatively, so for now I'm going with my gut and just taking a few of the "best" trades out of the handful. Overall, I'm really glad I went further on this. The boosting of the B/E strategies makes me trust my calculations on those more since they aren't so far from the passive management like they were with the raw data, and that really had me wondering what I did wrong.
What I will trade
Putting all this together, I am going to attempt to trade the following(demo for a bit to make sure I have the hang of it, then for keeps):
"System Details" I described above.
TP at -161.8%
Static SL at opposite side of confirmation candle - I won't move stops up to breakeven.
Trade only 7am-11am and 4pm-11pm signals.
Nothing where spread is more than 25% of trade width.
Looking at the data for these rules, test results are:
Adjusted Proft % (takes spread into account): 47.43%
I'll be sure to let everyone know how it goes!
Other Technical Details
ATR is only slightly elevated in this date range from historical levels, so this should fairly closely represent reality even after the COVID volatility leaves the scalpers sad and alone.
The sample size is much too small for anything really meaningful when you slice by hour or pair. I wasn't particularly looking to test a specific pair here - just the system overall as if you were going to trade it on all pairs with a reasonable spread.
Here's the spreadsheet for anyone that'd like it. (EDIT: Updated some of the setups from the last few days that have fully played out now. I also noticed a few typos, but nothing major that would change the overall outcomes. Regardless, I am currently reviewing every trade to ensure they are accurate.UPDATE: Finally all done. Very few corrections, no change to results.) I have some explanatory notes below to help everyone else understand the spiraled labyrinth of a mind that put the spreadsheet together.
I'm on the East Coast in the US, so the timestamps are Eastern time.
Time stamp is from the confirmation candle, not the indecision candle. So 7am would mean the indecision candle was 6:00-6:59 and the confirmation candle is 7:00-7:59 and you'd put in your order at 8:00.
I found a couple AM/PM typos as I was reviewing the data, so let me know if a trade doesn't make sense and I'll correct it.
Insanely detailed spreadsheet notes
For you real nerds out there. Here's an explanation of what each column means:
Pair - duh
Date/Time - Eastern time, confirmation candle as stated above
Win to -61.8%? - whether the trade made it to the -61.8% TP level before it hit the original SL.
Win to -100%? - whether the trade made it to the -100% TP level before it hit the original SL.
Win to -161.8%? - whether the trade made it to the -161.8% TP level before it hit the original SL.
Retracement level between -61.8% and -100% - how deep the price retraced after hitting -61.8%, but before hitting -100%. Be careful to look for the negative signs, it's easy to mix them up. Using the fib% levels defined in ParallaxFX's original thread. A plain hyphen "-" means it did not retrace, but rather went straight through -61.8% to -100%. Positive 100 means it hit the original SL.
Retracement level between -100% and -161.8% - how deep the price retraced after hitting -100%, but before hitting -161.8%. Be careful to look for the negative signs, it's easy to mix them up. Using the fib% levels defined in ParallaxFX's original thread. A plain hyphen "-" means it did not retrace, but rather went straight through -100% to -161.8%. Positive 100 means it hit the original SL.
Trade Width(Pips) - the size of the confirmation candle, and thus the "width" of your trade on which to determine position size, draw fib levels, etc.
Loser saved by 2 candle stop? - for all losing trades, whether or not the 2-candle stop loss would have saved the trade and how far it ended up getting if so. "No" means it didn't save it, N/A means it wasn't a losing trade so it's not relevant.
Spread(ThinkorSwim) - these are typical spreads for these pairs on ToS.
Spread % of Width - How big is the spread compared to the trade width? Not used in any calculations, but interesting nonetheless.
True Risk(Trade Width + Spread) - I set my SL at the opposite side of the confirmation candle knowing that I'm actually exposing myself to slightly more risk because of the spread(stop order = market order when submitted, so you pay the spread). So this tells you how many pips you are actually risking despite the Trade Width. I prefer this over setting the stop inside from the edge of the candle because some pairs have a wide spread that would mess with the system overall. But also many, many of these trades retraced very nearly to the edge of the confirmation candle, before ending up nicely profitable. If you keep your risk per trade at 1%, you're talking a true risk of, at most, 1.25% (in worst-case scenarios with the spread being 25% of the trade width as I am going with above).
Win or Loss in %(1% risk) including spread TP -61.8% - not going to go into huge detail, see the spreadsheet for calculations if you want. But, in a nutshell, if the trade was a win to 61.8%, it returns a positive # based on 61.8% of the trade width, minus the spread. Otherwise, it returns the True Risk as a negative. Both normalized to the 1% risk you started with.
Win or Loss in %(1% risk) including spread TP -100% - same as the last, but 100% of Trade Width.
Win or Loss in %(1% risk) including spread TP -161.8% - same as the last, but 161.8% of Trade Width.
Win or Loss in %(1% risk) including spread TP -100%, and move SL to breakeven at 61.8% - uses the retracement level columns to calculate profit/loss the same as the last few columns, but assuming you moved SL to 0% fib level after price hit -61.8%. Then full TP at 100%.
Win or Loss in %(1% risk) including spread take off half of position at -61.8%, move SL to breakeven, TP 100% - uses the retracement level columns to calculate profit/loss the same as the last few columns, but assuming you took of half the position and moved SL to 0% fib level after price hit -61.8%. Then TP the remaining half at 100%.
Overall Growth(-161.8% TP, 1% Risk) - pretty straightforward. Assuming you risked 1% on each trade, what the overall growth level would be chronologically(spreadsheet is sorted by date).
Based on the reasonable rules I discovered in this backtest:
Date range: 6/11-7/3
Adjusted Proft % (takes spread into account): 47.43%
Demo Trading Results
Since this post, I started demo trading this system assuming a 5k capital base and risking ~1% per trade. I've added the details to my spreadsheet for anyone interested. The results are pretty similar to the backtest when you consider real-life conditions/timing are a bit different. I missed some trades due to life(work, out of the house, etc), so that brought my total # of trades and thus overall profit down, but the winrate is nearly identical. I also closed a few trades early due to various reasons(not liking the price action, seeing support/resistance emerge, etc). A quick note is that TD's paper trade system fills at the mid price for both stop and limit orders, so I had to subtract the spread from the raw trade values to get the true profit/loss amount for each trade. I'm heading out of town next week, then after that it'll be time to take this sucker live!
Date range: 7/9-7/30
Adjusted Proft % (takes spread into account): 20.73%
Starting Balance: $5,000
Ending Balance: $6,036.51
Live Trading Results
I started live-trading this system on 8/10, and almost immediately had a string of losses much longer than either my backtest or demo period. Murphy's law huh? Anyways, that has me spooked so I'm doing a longer backtest before I start risking more real money. It's going to take me a little while due to the volume of trades, but I'll likely make a new post once I feel comfortable with that and start live trading again.
Summarizing some free trading idea resources I've been using
I've been following many free resources on youtube and twitter to generate trading ideas. Some of them are suspicious; some are more like boasting their wining trades but never post any losing trades. I see many people ask about trading ideas/resources, so I want to briefly share some resources I find useful. Twitter resources:
Instrument: Mostly SPX/SPY/ES
Highlights: TicTocTick is amazingly good at levels, spotting sellers and buyers levels. Everyday he posts his plan for the next day of the following format: If open above X, long/short bias, target Y. If open below X, short/long bias, target Z. Intraday he sometimes send "warnings" of potential big sellers / buyers at certain level. His price target and long/short bias is often right in my experience. His levels are useful for day trades IMHO.
Notes: (1) even with his plan, one needs an actionable plan. (2) He sometimes delete his tweets. His day-by-day and intraday tweets are more actionable than his longer term view. (3) he sometimes tweets political and controversial non-stock related things.
Trade transparency: 0/5 (doesn't post any trades)
Live update in-time: 5/5 (updates very frequently)
Actionable trading plan: 1/5 (good at levels and price targets. need your own plan)
Live interaction: 0/5 (no interaction)
Educational: 2/5 (can learn the technique from other resources. TicTock doesn't teach you directly)
Instrument: Mainly SPY/SPX/ES
Technique: candlestick patterns, Fib levels, support and resistance levels etc
Style: only day trading
Highlights: he diligently post daily plan and many educational resources, sometimes intraday updates. Had many good trades.
Notes: I haven't followed him long but so far so good. He also recently has educational youtube videos.
Trade transparency: X/5 (hard to measure)
Live update in-time: 2.5/5 (updates frequently)
Actionable trading plan: 3.5/5
Live interaction: X/5
Educational: 5/5 (youtube videos)
Technique: candlestick patterns, support and resistance levels, trendlines, channels etc
Instrument: SPX/SPY, Forex, Cryptocurrency,, Gold and Silver.
Style: holding for a few hours for SPX/SPY, swing trade for all
Timeframe: 8H for analysis. Lower time frame for entry.
Trading frequency: 1-2 trades per week.
Highlights: For SPX, he rode the big drop down in March; rode the rally up, and rode some pullbacks down in April. Got chopped in May. Now he's positinoning long. He also did well in Gold and Silverthis month. He only uses candle sticks, support and resistance lines, trendlines, and sometimes true trend indicator. He doesn't use volume though.
Youtube style: 2 videos every trading day: (1) live at 9am ET for 1-2 hours and talk about his plan and market analysis. Sometimes he trades during the live session (enter / exit). (2) after market closes he summarizes the day, and talks about plans for the next day. (3) Every weekend he gives out his technical analysis for the next week.
What I like: His levels on the chart are very good. He is also very transparent about his trades no matter whether it's winning or losing. He also explains the general economic environment.
Trade transparency: 4/5 (not knowing trading size; but knowing entry/exit)
Real-time update: 2.5/5 (two times a day)
Actionable trading plan: 5/5
Live interaction: 3.5/5 (some interaction on youtube live; Jordan responses to youtube comments)
Timeframe: all time frames. Mostly 5min, 1H, 1D, 1W, 1M.
Trading frequency: very frequent. multiple trades per day.
Highlights: Justin is very good at seeing through market maker manipulation and highly manipulated stocks. He often explained his plan and his outlook (especially in OPEX days) in his YouTube channel. The stocks on their weekly watchlist tend to do very well. He does live Q&A on youtube as well everyday where one can ask him to look at a chart.
Youtube style: Three videos by his team every trading day: (1) live at 9:30am ET; does 1-2 live scalping trades. Explains what he thinks of the market. (might discontinue) (2) at noon: summarizes what happened and what he sees is happening later in the day. Some of his trading plans. (3) 4:15pm ET: summarizes today and looking forward to the rest of the week. Videos (1) and (2) include live Q&A. I've asked many questions on youtube. Every weekend has two videos talking about plans for the next week.
What I like: The Q&A and Justin's outlook of the market, his team's stock pick.
The scalping trades in the morning is not very suitable for small accounts since they will trade for example 100 shares of BA (~160) to scalp a few dollars per share.
Even though the stocks on their weekly watchlist does well very, one still need to come up with an actionable plan. Very often say they recommend stock A on Sunday, and on Monday it already gaps up big. They sometimes do YOLO options -- big risk big rewards-- options can go to 0.
Besides the free content, everyone can get a free one-week trial for their paid membership, or a 2-week free trial by winning a lottery game on their youtube ( what I did) or knowing someone in their group and get a referral. What I like about the group: (i) very frequently updates each day on SPY and stocks on the watchlist. (ii) all their positions, Profit / Loss are very transparent. I learned a lot about how to manage trades by observing their live trades. (iii) There are many very experienced traders in the group posting their trading ideas, plans, entry/exit, and there are many live discussions. (iv) There's a "helpdesk" in the group where members' questions will be answered in minutes. I often ask about my trading plan, entries/ targets.
Trade transparency: 0/5 (free content: not knowing entry/exit nor position size);5+/5 (membership\*)*
Live update in-time: 3.5/5 (free content: three times a day);5+/5 (membership\*)*
Highlights: I follow their free Shadow trader swing newsletter, where every few days they post some trading ideas and analysis with actionable plan. Their twitter account will also real-time update their entry/exit and trade management.
What I like: I enjoyed learning what they look at to find a good set-up and how to manage a trade. They also have a spreadsheet tracking all their positions and profit/loss. All the winning/losing trades are transparent.
Notes: Because of the current market volatility, during certain weeks the swing trading performance is quite shaky. Profits (per 100K account with no more than 30K invested each time): 2020YTD: +9K, 2019: +6K; 2018: +30K; 2017: +3K; 2016: +2.5K; 2015: -1.8K.
Trade transparency: 5/5
Live update in-time: 5/5 (updates frequently)
Actionable trading plan: 5/5
Live interaction: 0/5 (newsletter and twitter alerts only)
Educational: 4.5/5 (the newsletter explains set-ups, what sectors they are looking at)
I've spent much time looking for free contents, and I like the ones above. Also looking forward to hearing about other good/bad resources. I might also update this post if there are enough interests. NFA
50 + Free & Best Selling Discounted Udemy: Business Analytics, Agile Project Management 200+ Tools with Kanban Scrum Devops , Big Data on Amazon web services (AWS), Front End Web Development , PHP MySQL & CodeIgniter Course: Complete Guide, Fundamentals of Engineering Thermodynamics and more
Business Analytics, Agile Project Management 200+ Tools with Kanban Scrum Devops , Big Data on Amazon web services (AWS), Front End Web Development , PHP MySQL & CodeIgniter Course: Complete Guide, Fundamentals of Engineering Thermodynamics
Crosspost: My first trading bot, now 4 months in development, started trading live last week and already gained 10%!
Backtest screenshot: https://solrac.prodibi.com/a/1jwk24gd54qyqxv/i/jdydmjj8wrrm725 Here's my original post: https://www.reddit.com/algotrading/comments/hd7e6c/my_first_algo_trading_bot_in_python_is_getting/?utm_source=share&utm_medium=ios_app&utm_name=iossmf Since then we've grown to a team of five people. We started trading live last week with a $100 test account on Binance Futures and gained 10% in our first week! Some amazing updates in the works: we are building this bot to connect to multiple exchanges via websocket in order to execute commands as fast as possible, and control them all through one web interface. This is a high velocity leveraged trading bot that uses 50x leverage and risks 5% of the wallet per trade. Soon we will implement dynamic leverage and position sizes based on key risk factors, like trading during range highs and lows. Beyond that, we also want to add different crypto markets, and maybe even forex eventually. Our very next target is Digitex Futures, the first totally commission free zero fee crypto exchange! We think this will be a game changer as fees make a huge impact on profitability. The current backtest, which is returning 900x over a 1 year 7 month period (with 100% of profits compounded) is viewable at cryptoravager dot com. I still need help to add Sharpe, equity, & drawdown indicators to the chart. Anyone have experience with the tradingview library? Please give me any feedback or advice! I'm one of those developers turned algo traders. I have 20 years experience in web application development, and only 1 year in trading and markets. Back in January I paid a pro trader good money to learn the strategy my bot is now using, which I used successfully by hand in March / April. That personal history plus the stellar backtest is what spurred us on to reach this point today.
No, the British did not steal $45 trillion from India
This is an updated copy of the version on BadHistory. I plan to update it in accordance with the feedback I got. I'd like to thank two people who will remain anonymous for helping me greatly with this post (you know who you are) Three years ago a festschrift for Binay Bhushan Chaudhuri was published by Shubhra Chakrabarti, a history teacher at the University of Delhi and Utsa Patnaik, a Marxist economist who taught at JNU until 2010. One of the essays in the festschirt by Utsa Patnaik was an attempt to quantify the "drain" undergone by India during British Rule. Her conclusion? Britain robbed India of $45 trillion (or £9.2 trillion) during their 200 or so years of rule. This figure was immensely popular, and got republished in several major news outlets (here, here, here, here (they get the number wrong) and more recently here), got a mention from the Minister of External Affairs & returns 29,100 results on Google. There's also plenty of references to it here on Reddit. Patnaik is not the first to calculate such a figure. Angus Maddison thought it was £100 million, Simon Digby said £1 billion, Javier Estaban said £40 million see Roy (2019). The huge range of figures should set off some alarm bells. So how did Patnaik calculate this (shockingly large) figure? Well, even though I don't have access to the festschrift, she conveniently has written an article detailing her methodology here. Let's have a look.
How exactly did the British manage to diddle us and drain our wealth’ ? was the question that Basudev Chatterjee (later editor of a volume in the Towards Freedom project) had posed to me 50 years ago when we were fellow-students abroad.
This is begging the question.
After decades of research I find that using India’s commodity export surplus as the measure and applying an interest rate of 5%, the total drain from 1765 to 1938, compounded up to 2016, comes to £9.2 trillion; since $4.86 exchanged for £1 those days, this sum equals about $45 trillion.
This is completely meaningless. To understand why it's meaningless consider India's annual coconut exports. These are almost certainly a surplus but the surplus in trade is countered by the other country buying the product (indeed, by definition, trade surpluses contribute to the GDP of a nation which hardly plays into intuitive conceptualisations of drain). Furthermore, Dewey (2019) critiques the 5% interest rate.
She [Patnaik] consistently adopts statistical assumptions (such as compound interest at a rate of 5% per annum over centuries) that exaggerate the magnitude of the drain
The exact mechanism of drain, or transfers from India to Britain was quite simple.
Drain theory possessed the political merit of being easily grasped by a nation of peasants. [...] No other idea could arouse people than the thought that they were being taxed so that others in far off lands might live in comfort. [...] It was, therefore, inevitable that the drain theory became the main staple of nationalist political agitation during the Gandhian era.
The key factor was Britain’s control over our taxation revenues combined with control over India’s financial gold and forex earnings from its booming commodity export surplus with the world. Simply put, Britain used locally raised rupee tax revenues to pay for its net import of goods, a highly abnormal use of budgetary funds not seen in any sovereign country.
The issue with figures like these is they all make certain methodological assumptions that are impossible to prove. From Roy in Frankema et al. (2019):
the "drain theory" of Indian poverty cannot be tested with evidence, for several reasons. First, it rests on the counterfactual that any money saved on account of factor payments abroad would translate into domestic investment, which can never be proved. Second, it rests on "the primitive notion that all payments to foreigners are "drain"", that is, on the assumption that these payments did not contribute to domestic national income to the equivalent extent (Kumar 1985, 384; see also Chaudhuri 1968). Again, this cannot be tested. [...] Fourth, while British officers serving India did receive salaries that were many times that of the average income in India, a paper using cross-country data shows that colonies with better paid officers were governed better (Jones 2013).
Indeed, drain theory rests on some very weak foundations. This, in of itself, should be enough to dismiss any of the other figures that get thrown out. Nonetheless, I felt it would be a useful exercise to continue exploring Patnaik's take on drain theory.
The East India Company from 1765 onwards allocated every year up to one-third of Indian budgetary revenues net of collection costs, to buy a large volume of goods for direct import into Britain, far in excess of that country’s own needs.
So what's going on here? Well Roy (2019) explains it better:
Colonial India ran an export surplus, which, together with foreign investment, was used to pay for services purchased from Britain. These payments included interest on public debt, salaries, and pensions paid to government offcers who had come from Britain, salaries of managers and engineers, guaranteed profts paid to railway companies, and repatriated business profts. How do we know that any of these payments involved paying too much? The answer is we do not.
So what was really happening is the government was paying its workers for services (as well as guaranteeing profits - to promote investment - something the GoI does today Dalal (2019), and promoting business in India), and those workers were remitting some of that money to Britain. This is hardly a drain (unless, of course, Indian diaspora around the world today are "draining" it). In some cases, the remittances would take the form of goods (as described) see Chaudhuri (1983):
It is obvious that these debit items were financed through the export surplus on merchandise account, and later, when railway construction started on a large scale in India, through capital import. Until 1833 the East India Company followed a cumbersome method in remitting the annual home charges. This was to purchase export commodities in India out of revenue, which were then shipped to London and the proceeds from their sale handed over to the home treasury.
While Roy's earlier point argues better paid officers governed better, it is honestly impossible to say what part of the repatriated export surplus was a drain, and what was not. However calling all of it a drain is definitely misguided. It's worth noting that Patnaik seems to make no attempt to quantify the benefits of the Raj either, Dewey (2019)'s 2nd criticism:
she [Patnaik] consistently ignores research that would tend to cut the economic impact of the drain down to size, such as the work on the sources of investment during the industrial revolution (which shows that industrialisation was financed by the ploughed-back profits of industrialists) or the costs of empire school (which stresses the high price of imperial defence)
Since tropical goods were highly prized in other cold temperate countries which could never produce them, in effect these free goods represented international purchasing power for Britain which kept a part for its own use and re-exported the balance to other countries in Europe and North America against import of food grains, iron and other goods in which it was deficient.
Re-exports necessarily adds value to goods when the goods are processed and when the goods are transported. The country with the largest navy at the time would presumably be in very good stead to do the latter.
The British historians Phyllis Deane and WA Cole presented an incorrect estimate of Britain’s 18th-19th century trade volume, by leaving out re-exports completely. I found that by 1800 Britain’s total trade was 62% higher than their estimate, on applying the correct definition of trade including re-exports, that is used by the United Nations and by all other international organisations.
While interesting, and certainly expected for such an old book, re-exporting necessarily adds value to goods.
When the Crown took over from the Company, from 1861 a clever system was developed under which all of India’s financial gold and forex earnings from its fast-rising commodity export surplus with the world, was intercepted and appropriated by Britain. As before up to a third of India’s rising budgetary revenues was not spent domestically but was set aside as ‘expenditure abroad’.
So, what does this mean? Britain appropriated all of India's earnings, and then spent a third of it aboard? Not exactly. She is describing home charges see Roy (2019) again:
Some of the expenditures on defense and administration were made in sterling and went out of the country. This payment by the government was known as the Home Charges. For example, interest payment on loans raised to finance construction of railways and irrigation works, pensions paid to retired officers, and purchase of stores, were payments in sterling. [...] almost all money that the government paid abroad corresponded to the purchase of a service from abroad. [...] The balance of payments system that emerged after 1800 was based on standard business principles.India bought something and paid for it.State revenues were used to pay for wages of people hired abroad, pay for interest on loans raised abroad, and repatriation of profits on foreign investments coming into India. These were legitimate market transactions.
Indeed, if paying for what you buy is drain, then several billions of us are drained every day.
The Secretary of State for India in Council, based in London, invited foreign importers to deposit with him the payment (in gold, sterling and their own currencies) for their net imports from India, and these gold and forex payments disappeared into the yawning maw of the SoS’s account in the Bank of England.
It should be noted that India having two heads was beneficial, and encouraged investment per Roy (2019):
The fact that the India Office in London managed a part of the monetary system made India creditworthy, stabilized its currency, and encouraged foreign savers to put money into railways and private enterprise in India. Current research on the history of public debt shows that stable and large colonies found it easier to borrow abroad than independent economies because the investors trusted the guarantee of the colonist powers.
Against India’s net foreign earnings he issued bills, termed Council bills (CBs), to an equivalent rupee value. The rate (between gold-linked sterling and silver rupee) at which the bills were issued, was carefully adjusted to the last farthing, so that foreigners would never find it more profitable to ship financial gold as payment directly to Indians, compared to using the CB route. Foreign importers then sent the CBs by post or by telegraph to the export houses in India, that via the exchange banks were paid out of the budgeted provision of sums under ‘expenditure abroad’, and the exporters in turn paid the producers (peasants and artisans) from whom they sourced the goods.
Sunderland (2013) argues CBs had two main roles (and neither were part of a grand plot to keep gold out of India):
Council bills had two roles. They firstly promoted trade by handing the IO some control of the rate of exchange and allowing the exchange banks to remit funds to India and to hedge currency transaction risks. They also enabled the Indian government to transfer cash to England for the payment of its UK commitments.
The United Nations (1962) historical data for 1900 to 1960, show that for three decades up to 1928 (and very likely earlier too) India posted the second highest merchandise export surplus in the world, with USA in the first position. Not only were Indians deprived of every bit of the enormous international purchasing power they had earned over 175 years, even its rupee equivalent was not issued to them since not even the colonial government was credited with any part of India’s net gold and forex earnings against which it could issue rupees. The sleight-of-hand employed, namely ‘paying’ producers out of their own taxes, made India’s export surplus unrequited and constituted a tax-financed drain to the metropolis, as had been correctly pointed out by those highly insightful classical writers, Dadabhai Naoroji and RCDutt.
It doesn't appear that others appreciate their insight Roy (2019):
K. N. Chaudhuri rightly calls such practice ‘confused’ economics ‘coloured by political feelings’.
Surplus budgets to effect such heavy tax-financed transfers had a severe employment–reducing and income-deflating effect: mass consumption was squeezed in order to release export goods. Per capita annual foodgrains absorption in British India declined from 210 kg. during the period 1904-09, to 157 kg. during 1937-41, and to only 137 kg by 1946.
If even a part of its enormous foreign earnings had been credited to it and not entirely siphoned off, India could have imported modern technology to build up an industrial structure as Japan was doing.
This is, unfortunately, impossible to prove. Had the British not arrived in India, there is no clear indication that India would've united (this is arguably more plausible than the given counterfactual1). Had the British not arrived in India, there is no clear indication India would not have been nuked in WW2, much like Japan. Had the British not arrived in India, there is no clear indication India would not have been invaded by lizard people, much like Japan. The list continues eternally. Nevertheless, I will charitably examine the given counterfactual anyway. Did pre-colonial India have industrial potential? The answer is a resounding no. From Gupta (1980):
This article starts from the premise that while economic categories - the extent of commodity production, wage labour, monetarisation of the economy, etc - should be the basis for any analysis of the production relations of pre-British India, it is the nature of class struggles arising out of particular class alignments that finally gives the decisive twist to social change. Arguing on this premise, and analysing the available evidence, this article concludes that there was little potential for industrial revolution before the British arrived in India because, whatever might have been the character of economic categories of that period,the class relations had not sufficiently matured to develop productive forces and the required class struggle for a 'revolution' to take place.
Yet all of this did not amount to an economic situation comparable to that of western Europe on the eve of the industrial revolution. Her technology - in agriculture as well as manufacturers - had by and large been stagnant for centuries. [...] The weakness of the Indian economy in the mid-eighteenth century, as compared to pre-industrial Europe was not simply a matter of technology and commercial and industrial organization. No scientific or geographical revolution formed part of the eighteenth-century Indian's historical experience. [...] Spontaneous movement towards industrialisation is unlikely in such a situation.
So now we've established India did not have industrial potential, was India similar to Japan just before the Meiji era? The answer, yet again, unsurprisingly, is no. Japan's economic situation was not comparable to India's, which allowed for Japan to finance its revolution. From Yasuba (1986):
All in all, the Japanese standard of living may not have been much below the English standard of living before industrialization, and both of them may have been considerably higher than the Indian standard of living. We can no longer say that Japan started from a pathetically low economic level and achieved a rapid or even "miraculous" economic growth. Japan's per capita income was almost as high as in Western Europe before industrialization, and it was possible for Japan to produce surplus in the Meiji Period to finance private and public capital formation.
The circumstances that led to Meiji Japan were extremely unique. See Tomlinson (1985):
Most modern comparisons between India and Japan, written by either Indianists or Japanese specialists, stress instead that industrial growth in Meiji Japan was the product of unique features that were not reproducible elsewhere. [...] it is undoubtably true that Japan's progress to industrialization has been unique and unrepeatable
So there you have it. Unsubstantiated statistical assumptions, calling any number you can a drain & assuming a counterfactual for no good reason gets you this $45 trillion number. Hopefully that's enough to bury it in the ground. 1. Several authors have affirmed that Indian identity is a colonial artefact. For example seeRajan 1969:
Perhaps the single greatest and most enduring impact of British rule over India is that it created an Indian nation, in the modern political sense. After centuries of rule by different dynasties overparts of the Indian sub-continent, and after about 100 years of British rule, Indians ceased to be merely Bengalis, Maharashtrians,or Tamils, linguistically and culturally.
But then, it would be anachronistic to condemn eighteenth-century Indians, who served the British, as collaborators, when the notion of 'democratic' nationalism or of an Indian 'nation' did not then exist.[...]Indians who fought for them, differed from the Europeans in having a primary attachment to a non-belligerent religion, family and local chief, which was stronger than any identity they might have with a more remote prince or 'nation'.
Chakrabarti, Shubra & Patnaik, Utsa (2018). Agrarian and other histories: Essays for Binay Bhushan Chaudhuri. Colombia University Press Hickel, Jason (2018). How the British stole $45 trillion from India. The Guardian Bhuyan, Aroonim & Sharma, Krishan (2019). The Great Loot: How the British stole $45 trillion from India. Indiapost Monbiot, George (2020). English Landowners have stolen our rights. It is time to reclaim them. The Guardian Tsjeng, Zing (2020). How Britain Stole $45 trillion from India with trains | Empires of Dirt. Vice Chaudhury, Dipanjan (2019). British looted $45 trillion from India in today’s value: Jaishankar. The Economic Times Roy, Tirthankar (2019). How British rule changed India's economy: The Paradox of the Raj. Palgrave Macmillan Patnaik, Utsa (2018). How the British impoverished India. Hindustan Times Tuovila, Alicia (2019). Expenditure method. Investopedia Dewey, Clive (2019). Changing the guard: The dissolution of the nationalist–Marxist orthodoxy in the agrarian and agricultural history of India. The Indian Economic & Social History Review Chandra, Bipan et al. (1989). India's Struggle for Independence, 1857-1947. Penguin Books Frankema, Ewout & Booth, Anne (2019). Fiscal Capacity and the Colonial State in Asia and Africa, c. 1850-1960. Cambridge University Press Dalal, Sucheta (2019). IL&FS Controversy: Centre is Paying Up on Sovereign Guarantees to ADB, KfW for Group's Loan. TheWire Chaudhuri, K.N. (1983). X - Foreign Trade and Balance of Payments (1757–1947). Cambridge University Press Sunderland, David (2013). Financing the Raj: The City of London and Colonial India, 1858-1940. Boydell Press Dewey, Clive (1978). Patwari and Chaukidar: Subordinate officials and the reliability of India’s agricultural statistics. Athlone Press Smith, Lisa (2015). The great Indian calorie debate: Explaining rising undernourishment during India’s rapid economic growth. Food Policy Duh, Josephine & Spears, Dean (2016). Health and Hunger: Disease, Energy Needs, and the Indian Calorie Consumption Puzzle. The Economic Journal Vankatesh, P. et al. (2016). Relationship between Food Production and Consumption Diversity in India – Empirical Evidences from Cross Section Analysis. Agricultural Economics Research Review Gupta, Shaibal (1980). Potential of Industrial Revolution in Pre-British India. Economic and Political Weekly Raychaudhuri, Tapan (1983). I - The mid-eighteenth-century background. Cambridge University Press Yasuba, Yasukichi (1986). Standard of Living in Japan Before Industrialization: From what Level did Japan Begin? A Comment. The Journal of Economic History Tomblinson, B.R. (1985). Writing History Sideways: Lessons for Indian Economic Historians from Meiji Japan. Cambridge University Press Rajan, M.S. (1969). The Impact of British Rule in India. Journal of Contemporary History Bryant, G.J. (2000). Indigenous Mercenaries in the Service of European Imperialists: The Case of the Sepoys in the Early British Indian Army, 1750-1800. War in History
Forex news from ForexLive. The fastest Foreign Exchange market reporting and analysis. Live Forex and economic news. Technical analysis, headlines, Live quotes. The forex live rates are closely monitored by the FX Leaders team and are used for analysis as well as in the development of trading signals.In order to stay current, live rates are streamed directly from the market, updated every five seconds. A combination of data from the world’s leading liquidity providers, forex live rates are a great place to start when looking for an accurate quote. Forex Economic Calendar Live Update Today - Non-Farm Payroll, Factory Orders, etc. Fed, ECB, BoJ, BoE, SNB, BoC, RBA, RBNZ View live forex rates and prices for commodities, indices and cryptos. Live streaming allows you to quickly spot any changes to a range of market assets. Forex technical analysis in real-time from ForexLive. Technical trading analysis on currencies including charts on foreign exchange. Foreign exchange analysis. Welcome to Forex Cafe Trading Updates! Forex Cafe - Live quotes, stock charts and expert trading ideas. Forex Cafe is a social network for traders and investors on Stock, Futures and Forex markets! News. Forex Cafe for Traders (30 October 2019 17:51) All Trader Needs!! اخر الاحداث الاقتصادية . Metals. Platinum discuss (Visits: 222) Silver discuss (Visits: 152) Gold discuss ... Real-time forex news and the latest trading updates. What you need to know now about the GBP, Dollar, Yen, Euro and minors. Forex traders buy a currency pair if they think the exchange rate will rise and sell it if they think the opposite will happen. The Forex market remains open around the world for 24 hours a day with the exception of weekends. Let's see all available forex signals bitcoin app. More than 200 of them are presented here. Forex apps, The default setting shows prices in USD and sorts forex assets based on the market capitalization.
*LIVE* Crypto/Forex Update - 10/3 Bitnamix, Power of 2 ...
GreenBack Forex Group Whatsapp +27651227593 Call +27670127481 Fundamental Trading strategies Technical Trading strategies Lifetime Mentorship ️ Lifetime signals ️ Learn how to trade forex ... Free Education & Signals: 👉 https://bit.ly/freeroninsignals 👈 Free Facebook Group: https://bit.ly/RoninFacebook HUGE moves in the Dollar today that correlate... *LIVE* Crypto/Forex Update 10/3 Bitnamix, Power of 2 & EPIC Trading Launches! Bitnamix - Request our Next Team Link Form for Our Team Build! https://www.epic... LIVE FOREX TRADING #FOREXTRADING JOIN US DAILY HERE: https://www.imarketslive.com/corp/cjoin?enroller=ffxacademy BOOK YOUR FREE CALL WITH PATRICK TODAY! http... Enjoying the free content here on my YT channel? You'll probably enjoy my free training as well! I structure the entire process for how to get and stay profi... *LIVE* Crypto/Forex Update 10/9 Bitnamix, Power of 2 & EPIC Trading Launches! Bitnamix - Our EPIC Lifestyle Team Build System Gets YOU Personal PAID Member...