Is python coding profitable

Cheap Weikersheim (Baden-Wuerttemberg)

Mastering Python for Finance Forex Trading with OANDA API In the previous sections we implemented a trading system by connecting to Interactive Brokers Trader WorkStation X via the socket connections via a single port. However, many other brokers offer various ways of connecting custom trading software through an API. In this section we will learn how we can connect our trading strategy with OANDA's REST API. OANDA is a major player in the foreign exchange (forex) business service retail investors. We use a trend-following strategy to trade forex products. What does REST mean REST stands for Representational State Transfer. It refers to web service APIs for transferring data over HTTP using the GET. PUT. POST. Or DELETE methods. With the REST API we can stream. The best content for your career. Discover unlimited learning on demand for around 1 / day. Machine Learning and Pattern Recognition for Algorithmic Forex and Stock Trading Introduction Machine learning in any form, including pattern recognition, of course, has many uses from voice and face recognition, medical research. In this case, our question is whether we can use pattern recognition to relate previous situations that were similar in patterns. If we can do that, we can then base trades on what we know has happened with these patterns in the past and actually doing a profit would completely completely code everything by ourselves. If you enjoy this topic, the next step would be to look into GPU acceleration or threading. I just need Matplotlib (for data visualization) and some NumPy (for number crunching), and the rest is up to us. Python is, of course, a single-threaded language, which means that each script only uses a single CPU (usually this means it uses a single CPU core, and sometimes even half or a quarter or worse than that core). This is why programs in Python can take a while to computer something, but their processing might only be 5 and RAM 10. To learn more about threading, you can check out the threading tutorial on this website. The easiest way to get these modules these days is to use pip install. Pip is probably the easiest way to install packages when installing Python you should be able to type your command prompt like cmd. exe on windows or bash on linux and type: pip install numpy pip install matplotlib If you have problems still no problem, theres a tutorial for it: pip install python modules tutorial. If you still have problems, please do not hesitate to contact us using the contact details in the footer of this website. The plan is to take a group of prices in a time frame and convert them into percent change in an effort to normalize the data. Let's say we take 50 consecutive price points for the sake of explanation. What is good is to map this pattern into memory, move forward one price point, and reassign the pattern. For each pattern we map into memory, let's want to jump forward a bit, say 10 price points, and log where the price is at that point. Then we assign this result to the pattern and continue. Every pattern has its result. Next we take the current pattern and compare it to all previous patterns. Which is good, compare the percentage similarity to all of the previous patterns. If their percentage similarity is more than a certain threshold then they would think about it. From here we have maybe 20-30 comparable patterns from history. With these similar patterns, we can then aggregate all of their results, and come up with an estimated average result. With this average result, if it is very cheap, then we could initiate a purchase. If the result is not favorable, maybe we will sell or shortly. For the visualization, Heres is an example: In the example above, the predicted average pattern is to increase so that we could initiate a purchase. This series isn't going to end with you with any kind of get-rich-quick algorithm. There are a couple of known bugs with this program, and the chances of you being able to execute trades fast enough with this tick data is unlikely unless you are a bank. The goal here is to show you how simple and basic pattern recognition is. As long as you have some basic Python programming knowledge, you should be able to follow along. OANDA API Trading Utilities in Python Sample Programs Trading with the OANDA API through Python2.7 This repo contains a trading program that executes trades when WMA and SMA cross. There is also a file that contains some extremely simple functions that open a trade or an order. Clone this repo to the location of your choice Change api - ror. py to do what you want or just run api-trade-averages. py with Python2.7. To run the script please include the number of candles, WMA and SMA, the candlestick granularity, the instrument and your invoice. This script uses the sandbox environment, so please use Sandbox accountId. Python api-trade-averages. py 10 S5 EURUSD This program is intended to demonstrate OANDA API functionality and is not intended as investment advice or a solution for buying or selling investment products. QSForex is an open source event-driven backtesting and live trading platform for use in the currency markets, currently in an alpha state. It was created as part of the Forex Trading Diary series on QuantStart to provide the systematic trading community with a robust trading engine that enables easy forward strategy implementation and testing. The software is provided under an approved MIT license (see below). Open Source - QSForex has been released under a highly legal open source MIT license that allows full use in both research and commercial applications, without limitation, but without any warranty. Free - QSForex is completely free and does not cost anything to download or use. Collaboration - Since QSForex is open source, many developers work together to improve the software. New features are added frequently. All errors are quickly identified and corrected. Software Development - QSForex is written in the Python programming language for easy cross-platform support. QSForex includes a suite of unit tests for most of its calculation code, and new tests are constantly being added for new functionality. Event Driven Architecture - QSForex is fully event driven for both backtesting and live traders, resulting in a direct transition of strategies from a research / test phase to a live trading implementation. Transaction Costs - Spread costs are included by default for all BackTest strategies. Backtesting - QSForex offers intraday tick resolution multi-day multi-currency pair backtesting. Trading - QSForex currently supports live intraday trading using the OANDA Brokerage API via a portfolio of pairs. Performance Metrics - QSForex currently supports basic performance measurement and equity visualization via the Matplotlib and Seaborn visualization libraries. Installation and Use 1) Visit www. oanda / and set up an account to get the API authentication credentials you need to trade live. I explain how to do this in this article: www. quantstart / articles / Forex-Trading-Diary-1-Automated-Forex-Trading-with-the-OANDA-API. 2) Clone this git repository in a convenient location on your computer using the following command in your terminal: git clone github / mhallsmoore / qsforex. git. Alternatively, you can download the ZIP file of the current master branch at github / mhallsmoore / qsforex / archive / master. zip download. 3) Create a set of environment variables for all the settings that are in the settings. py were found in the application's root directory. Alternatively, you can change your specific settings by overriding the calls to os. environ. get (.) Set for each setting: 4) Create a virtual environment (virtualenv) for the QSForex code and use pip to install the requirements. For example on a Unix-based system (Mac or Linux) you can create such a directory as follows by entering the following commands in the terminal: This will create a new virtual environment for installing the packages. Assuming you have copied the QSForex git repository into an example directory like / projects / qsforex / (change this directory below where you installed QSForex) then to install the packages you need to run the following commands: This will take some time to be put together as NumPy, SciPy, Pandas, Scikit-Learn and Matplotlib. There are many packages that are required to do this job, so take a look at these two articles for more information: You will also need to create a symbolic link from your site package directory to your QSForex installation directory in order to be able to its to call import qsforex inside the code. To do this, you need a command that is similar to the following: Make sure you change / projects / qsforex to your installation directory and /venv/qsforex/lib/python2.7/site-packages/ to the virtualenv site packages directory. You can now correctly execute the following commands. 5) At this stage, if you just want to practice or trade live then you can use Python trading / trading. py run. Which will use the standard strategy of TestStrategy. This simply buys or sells a currency pair every 5th tick. It's just for testing - don't use it in a live trading environment. If you have a useful strategy to create then just create a new class with a descriptive name, e.g. MeanReversionMultiPairStrategy and make sure it has a calculatesignals method. You need to pass this class to list the pairs as well as queue the events as they are in trading / trading. py. See strategy / strategy. py for details. 6) In order to carry out the backtesting, it is necessary to generate simulated forex data or download historical tick data. If you just want to try out the software, the fastest way to generate a sample backtest is to generate some simulated data. The current data format used by QSForex is the same as that used by the Dukascopy Historical Data Feed at www. dukascopy / Swiss / English / Market / historical / provided. To generate some historical data, make sure the CSVDATADIR setting in settings. py is set to a directory where you want the historical data to live. You then need to generateimulatedpair. py run. Which is located in the scripts / directory. It expects a single line of argument to command, which in this case is the currency pair in BBBQQQ format. Example: At this stage the script is programmed to create a single month of data for January 2014, which means that you can see single files of the format BBBQQQYYYYMMDD. csv (e.g. GBPUSD20140112.csv) appear in your CSVDATADIR for all working days in this month. If you want to change the month / year of the data output, just change the file and start. 7) After the historical data has been generated, it is possible to carry out a backtest. The backtest file itself is stored in backtest / backtest. py saved. But that only includes the backtest class. In order to run a backtest, you have to instantiate this class and supply it with the necessary modules. Moving Average Crossover Implementation in the examples / mac. py file The best way to see how this happens is to look at the example and use this as a template. This uses the MovingAverageCrossStrategy, which is included in the strategy / strategy. py is found. This preset applies to trading GBP / USD and EUR / USD to display multiple currency pairs. It uses data found in CSVDATADIR. To run the sample back test, just do the following: This will take some time. On my Ubuntu desktop system at home, with the historical data generated via generatesimulatedpair. py were generated. It takes about 5-10 minutes to run. Much of this calculation occurs at the end of the actual backtest when the loss is calculated, so please keep in mind that the code has not hung up for completion Please leave. 8) If you want to see the performance of the back test you can just use output. use py to show an equity curve, period returns (i.e. tick-to-tick returns) and a drawdown curve: and that's it. At this stage, you are ready to create your own backtests by using strategies in strategy / strategy . py or append it and use the data downloaded from DukasCopy (www. dukascopy / swiss / english / marketwatch / historical /). If you have any questions about the installation, please feel free to email me at mikequantstart. If you have any bugs or other issues that you think may be specific to the code base, feel free to open a github issue here: github / mhallsmoore / qsforex / questions Copyright (c) 2015 Michael Hallen-Moore This is the Permission granted to the summons to obtain a copy of this software and the associated documentation files (the software) for persons to work in the software without limitation, including without limitation the rights to use, copy, modify, merge, to publish, distribute, further license and / or sell copies of the software; and to make the software available to persons to whom the software is to be permitted subject to the following conditions: the above copyright notice and this permission in all copies or substantial parts will be included in the software. The software is deemed to be, without warranty of any kind, DIRECT OR INDIRECT, INCLUDING, BUT NOT, WARRANTIES OF FITNESS FOR A PARTICULAR PURPOSE AND NON-INFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDER BE LIABLE FOR DAMAGE OR OTHER LIABILITY, IN ANY CONTRACT, DEBT, OR OTHERWISE, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR USE OR ANY OTHER IN THE SOFTWARE. Forex Trading Disclaimer Forex trading on margin carries a high level of risk and may not be suitable for all investors. Past performance is not an indication of future results. The high degree of leverage can work against you as well as for you. Before deciding to invest in forex, you should carefully consider your investment goals, experience and risk tolerance. There is a possibility that you will get a loss of some or all of your initial investment and therefore you should not invest money that you cannot afford to lose. You should be aware of all of the risks associated with forex trading and seek advice from an independent financial advisor if you have any doubts. Forex Trading Diary 1 - Automated Forex Trading with the OANDA API By Michael Halls-Moore on January 21, 2015 I mentioned in the QuantStart: 2014 report article that I would spend some 2015 spending on automated forex trading. Given that I usually research myself into stock and futures markets, I thought it would be fun (and educational) to write diary style about my experience of entering the forex market. Each journal entry will try to build on all of those, but should also be relatively self-contained. In this first entry of the diary I'll describe how to set up a new practice brokerage account with OANDA as well as how to create a basic multithreaded event driven trading engine that can be run automatically, both in a practice and live setting. Last year we spent a lot of time looking at the event driven backtester. Especially for stocks and ETFs. The one I have here below is geared towards Forex and can be used for both paper trading or live trading. I have all of the following instructions for Ubuntu 14.04, but they should easily translate to Windows or Mac OS X, using a Python distribution like Anaconda. The only additional library used for the Python trading machine is the requirements library, which is required for HTTP communication to the OANDA API. Since this is the first item directly about Forex trading and the code below can be adapted directly to a live trading environment, I would like to introduce the following disclaimers: Disclaimer: Trading Forex at margin carries a high level of risk, and may not suitable for all investors. Past performance is not an indication of future results. The high degree of leverage can work against you as well as for you.Before deciding to invest in forex, you should carefully consider your investment goals, experience and risk tolerance. There is a possibility that you will get a loss of some or all of your initial investment and therefore you should not invest money that you cannot afford to lose. You should be aware of all of the risks associated with forex trading and seek advice from an independent financial advisor if you have any doubts. This software is provided "as is" and all warranties, expressed or implied, including but not limited to the implied warranties of merchantability and fitness for a particular purpose, are excluded. In no event shall the Regents or contributors be liable for any direct, indirect, incidental, special, exemplary, or consequential damages (including, but not limited to, procurement of replacement goods or loss of service, data or profits, or business interruption) arising in any event from the use of these Software has arisen, even if the possibility of such damage is pointed out, on any liability theory, be it contractual, negligent or tort (including negligence or otherwise). Setting up an account with OANDA The first question that comes to mind is why choose OANDA. Simply put, after a little googling around for forex brokers that had APIs, I saw that OANDA recently had a proper REST API that could be easily communicated with almost any language in an extremely straightforward manner. After reading through their developer API documentation. I decided to give them a try, at least with a practice account. To be clear - I have no previous or existing relationship with OANDA and am only providing this recommendation based on my limited experience playing around with their practice API and some brief usage (for market data download) while previously employed with a fund. If anyone has come across any other forex brokers that also have a similar modern API then I'd be happy to give them a look too. Before using the API you have to register for a practice account. To do this, go to the registration link. You will then see the following screen: You can then log in with your credentials. Be sure to select the fxTradePractice tab from the login screen: once you will need to take a note of your account number. It is listed below the black My Funds header next to Primary. Mine is a 7-digit number. In addition, you also need to generate a personal API token. To do this, click on Manage API access in the lower left under the Other Actions tab: At this stage, you can generate an API token. You'll need the key for use later, so make sure to write it down too. You will now start the FXTrade Practice application which will allow us to see the executed orders and our (paper) profit & loss. If you're running an Ubuntu system, you'll need to install a slightly different version of Java. In particular the Oracle version of Java 8. If you do not do this, the practice simulator will not load from the browser. I ran these commands on my system: you will now be able to start the practice trading environment. Return to the OANDA Dashboard and click the Start FXTrade Practice link highlighted in green. A Java dialog will ask if you want to run it. Click Run and the fxTrade practice program will load. Mine has been set to a 15 minute candlestick chart of EUR / USD with the counting panel on the left: at this point we are ready to start designing and coding our automated forex trading system against the OANDA API. Trading Architecture Overview If you've followed the event-driven series of backtesters for stocks and ETFs that I created last year, you will be well aware of how such an event-driven trading system works. For those of you who are new to event-driven software. I would strongly suggest reading through the article for some insight into how they work. Essentially, the entire program is run in an infinite while loop that only ends when the trading system is switched off. The central communication mechanism of the program takes place via a queue that contains events. The queue is constantly polled to look for new events. As soon as an event has been taken from the queue, it must be handled by an appropriate component of the program. A market data feed could therefore generate TickEvents, which are placed on the queue when a new market price arrives. A signal-generating strategy object can generate OrderEvent-S, which are to be sent to a brokerage. The usefulness of such a system is given by the fact that it does not matter what order or types of events are queued as they will always be handled correctly by the correct component within the program. In addition, different parts of the program can run on separate threads. Which means that there is never any waiting for a particular component before processing any other. This is extremely useful in algorithmic trading situations where market data feed handlers and strategy signal generators have very different performance characteristics. The main trading loop is given by the following Python pseudocode: As mentioned above, the code runs in an infinite loop. First the queue is queried to get a new event. If the queue is empty, the loop will simply restart after a short period of sleep called a heartbeat. If an event is found, its type is evaluated and then the relevant module (either the strategy or the execution handler) is called to handle the event and possibly generate new ones that are put back in the queue. The basic components that we will create for our trading system include the following: Streaming Price Handler - This will keep a long term connection open to OANDAs servers and send tick data (i.e. bid / ask) over the connection for any instruments interested were in. Strategy Signal Generator - This will take a sequence of tick events and use them to generate trade orders that will be executed by the execution handlers. Execution Handler - takes a series of order events and then blindly executes them with OANDA. Events - These objects represent the messages that are passed on the event queue. For this implementation we only need two, namely the TickEvent and the OrderEvent. Main Entry Point - The Main Entry Point also contains the trade loop which continuously polls the message queue and sends messages to the correct component. This is often referred to as an event or event handler. We will now discuss the implementation of the code in detail. At the bottom of the article is the full listing of all source code files. If you put them in the same directory and python trading. py, you will start generating orders, provided you have filled in your OANDA account number and authentication token. Python Implementation It is bad to store passwords or authentication keys in a code base because you can never predict who will eventually have access to a project. In a production system, we would store these credentials as environment variables with the system and then query these envvars every time the code is redeployed. This ensures that passwords and auth tokens are never stored in a version control system. However, since we are only interested in developing a toy trading system and not dealing with production details in this article, we will instead separate these autotoken into a settings file. In the following settings. py configuration file we have a dictionary called ENVIRONMENTS which stores the API endpoints for the OANDA streaming API and the trading API. Each sub-dictionary contains three separate API endpoints: real. Practice and sandpit. The sandbox API is purely for testing code and checking that there are no errors or bugs. It doesn't have the uptime guarantees of the real-world or real-world APIs. The practice API, essentially, provides the ability to trade paper. This means that it offers all the features of the real API on a simulated practice account. The real API is just that - it's live trading. If you use this endpoint in your code, it will trade against your live account balance. VERY EXTREMELY CAREFULLY IMPORTANT: When trading against the Practice API remember that an important transaction is cost that of impact on the market. Is not taken into account. Since no trades are actually placed in the environment, these costs have to be offset against a market impact model if you want to realistically assess the performance. In the following we use the practice account specified in the DOMAIN setting. We need two separate dictionaries for the domains, one each for the streaming and trading API components. Finally we have the ACCESSTOKEN and ACCOUNTID. I've filled the two below with dummy IDs, so you'll need to use your own which can be accessed from the OANDA account page: the next step is to queue the events that the events are being used to help out each one Define components to communicate. We need two: TickEvent and OrderEvent. The first stores information on instrument market data such as the (best) supply / demand and trading time. The second is used to submit orders to the Execution Processor and thus contains the instrument, the number of units to trade, the order type (market or limit) and the side (i.e. buy and sell). To future proof our event code, we're going to create a base class called Event and inherit all events from it. The code is shown below in events. py: The next class we are going to create is going to cover trading strategy. In this demo we're going to create a pretty nonsensical strategy that just gets all the market ticks and randomly buys or sells 10,000 units of EUR / USD every 5th tick. Sure, this is a ridiculous strategy, but it's fantastic for testing because it's easy to code and understand. In future journal entries, we'll be replacing this with something more exciting that (hopefully) makes a profit. The strategy. py file can be found below. Lets work through them and see what's going on. First we import the random library and the OrderEvent object from events. py. We need the occasional lib to choose an occasional buy or sell order. We need OrderEvent because the strategy object sends orders to the event queue which is later executed by the execution handler. The TestRandomStrategy class takes the device (in this case EUR / USD), the number of units, and the event queue as the set of parameters. It then creates a ticks counter that is used to tell how many TickEvent instances it has seen. Most of the work is done in the calculatesignals method, which simply takes an event, determines if it's a TickEvent (ignore otherwise), and increments the tick counter. Then it checks whether the count is divisible by 5 and then buys or sells the specified number of units at random with a market order. Its certainly not the world's greatest trading strategy, but it will be more than suitable for our OANDA Brokerage API testing purposes. The next component is the execution handler. The task of this class is to react to OrderEvent instances and to mute requests to the broker (in this case OANDA). That is, there is no risk management or portfolio construction overlay. The execution handler just executes any order. We need to pass all authentication information to the Execution class, including the domain (practice, real, or sandbox), access token, and account ID. Then we create a secure connection with httplib. One of the pythons built in libraries. Most of the work is done in executeorder. The method requires an event as a parameter. It then constructs two dictionaries - the headers and the params. These dictionaries are then correctly encoded (in part through another Python library urllib) as an HTTP POST request sent to OANDA's API. We pass the Content-Type and Authorization header parameters, which include our authentication information. In addition, we encode the parameters that define the instrument (EUR / USD), units, order type and side (buy / sell). Finally, we make the request and save the answer: the most complex component of the trading system is the StreamingForexPrices object, which handles OANDA's market price updates. There are two methods: connecttostream and streamtoqueue. The first method uses the Python request library to connect to a streaming socket with the appropriate headers and parameters. The parameters include the account ID and the necessary instrument list that can be heard for updates (in this case only EUR / USD). Notice the following line: This will stream the connection and keep it open for that long. The second method, streamtoqueue. Actually trying to connect to the stream. If the response is unsuccessful (i.e. the response code is not HTTP 200) then we just come back and exit. If it succeeds, we try to load the JSON package returned into a Python dictionary. Finally, we convert the Python dictionary with the instrument, bid / ask and timestamp into a TickEvent which is sent to the event queue: we now have all the important components in place. The final step is to put everything we've written so far into a main program. The destination of this file, known as trading. py. Is to create two separate threads. One of which runs the price price and the other runs the trade handler. Why do we need to put two separate threads simply, we run two separate pieces of code, both of which run continuously. If we were to create a threadless program, the streaming socket used for price updates would never re-open the main code path and therefore we would never do any trading. Similarly, if we ran the trading loop (see below), we would actually never return the flow path to the price streaming pedestal. So we need multiple threads, one for each component, so that they can run independently. They communicate with each other through the queue. Let's examine this a little further. We create two separate threads with the following lines: We pass the function or method name to the keyword argument of the target, and then pass an iterable (like a list or a tuple) to the args keyword argument that passes these arguments to the actual Method passes / function. Finally we start both threads with the following lines: So we are able to run two, effectively infinite loops, code segments independently, both of which communicate via the event queue. Note that the Python threading library does not create a multithreaded multicast environment due to Python's implementation of CPython and the Global Interpreter Lock (GIL). If you want to read more about multithreading on Python, take a look at this article. Lets examine the rest of the code in detail. First we import all the necessary libraries including the queue. Thread and time. We then import all of the above code files. Personally, I prefer enabling configuration settings, which is a habit I picked up from working with Django. After that we define the trading function which was explained in the Python pseudocode above. An infinite while loop is performed (while True :) which continuously polls the event queue and only skips the loop if it is empty. When an event is found it is either a TickEvent or an OrderEvent and then the appropriate component is called to execute it. In this case, it is either a strategy or an execution handler. The loop then simply sleeps for heartbeat seconds (in this case 0.5 seconds) and continues. Finally, we define the main entry point of the code in the main function. It's well commented on below, but I'll summarize here. Essentially, we instantiate the event queue and define the instruments / units. Then we create the StreamingForexPrices-Price-Streaming-Class and then the Execution-Execution-Procedure. Both will receive the required authentication information, which OANDA will provide when creating an account. Then we create the TestRandomStrategy instance.Finally we define the two threads and then start them: To execute the code, you simply have to put all the files in the same directory and call the following on the terminal: Note that to stop the code at this stage, a severe abort of the Python process. Via Ctrl-Z or the equivalent I've not had an extra thread to handle looking for the sys. exit (), which would be required to safely stop the code. One way to stop the code on an Ubuntu / Linux machine is to type: Then pass the output of this (a process number) into the following: Where PROCESSID needs to be replaced with the output from pgrep. Note that this is not particularly good. In later articles, we will create a more sophisticated stop / start mechanism that takes advantage of Ubuntus process monitoring to allow the trading system to run 24/7. The output after 30 seconds or so, depending on the time of day relative to the main trading hours for EUR / USD for the above code, is given below: The first five lines show the JSON tick data prices returned by OANDA with Bid / Ask. You can then see the execution order output as well as the JSON response returned by OANDA confirming the opening of a buy trade for 10,000 EUR / USD units and the price obtained. This will continue indefinitely until you exit the program with a Ctrl-Z command or something similar. Whats Next In later articles we will be making some much needed improvements including: Real Strategies - Proper Forex strategies that generate profitable signals. Production infrastructure - remote server implementation and 24/7 monitored trading system with stop / start capability. Portfolio and risk management portfolio and risk overlays for all proposed orders from the strategy. Multiple Strategies - Building a Portfolio of Strategies that Integrate into the Risk Management Overlay As with the equities event-driven backtesters, we also need to create a forex backtesting module. This will allow us to research quickly and make it easier to employ strategies. Full code settings. py (don't forget to change ACCOUNTID and ACCESSTOKEN):