adelorenzo/stable
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity

merging develop into async. requirement.txt conflict resolved

Browse Source
This commit is contained in:
misagh 2018-09-06 20:28:07 +02:00
commit 13ffd88053
9 changed files with 235 additions and 99 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

7
docs/configuration.md
View File

@ -23,11 +23,12 @@ The table below will list all configuration parameters.
| `ticker_interval` | [1m, 5m, 30m, 1h, 1d] | No | The ticker interval to use (1min, 5 min, 30 min, 1 hour or 1 day). Default is 5 minutes
| `fiat_display_currency` | USD | Yes | Fiat currency used to show your profits. More information below.
| `dry_run` | true | Yes | Define if the bot must be in Dry-run or production mode.
| `process_only_new_candles` | false | No | If set to true indicators are processed only once a new candle arrives. If false each loop populates the indicators, this will mean the same candle is processed many times creating system load but can be useful of your strategy depends on tick data not only candle. Can be set either in Configuration or in the strategy.
| `minimal_roi` | See below | No | Set the threshold in percent the bot will use to sell a trade. More information below. If set, this parameter will override `minimal_roi` from your strategy file.
| `stoploss` | -0.10 | No | Value of the stoploss in percent used by the bot. More information below. If set, this parameter will override `stoploss` from your strategy file.
| `trailing_stoploss` | false | No | Enables trailing stop-loss (based on `stoploss` in either configuration or strategy file).
| `trailing_stoploss_positve` | 0 | No | Changes stop-loss once profit has been reached.
| `trailing_stoploss_positve_offset` | 0 | No | Offset on when to apply `trailing_stoploss_positive`. Percentage value which should be positive.
| `trailing_stop` | false | No | Enables trailing stop-loss (based on `stoploss` in either configuration or strategy file).
| `trailing_stop_positve` | 0 | No | Changes stop-loss once profit has been reached.
| `trailing_stop_positve_offset` | 0 | No | Offset on when to apply `trailing_stop_positive`. Percentage value which should be positive.
| `unfilledtimeout.buy` | 10 | Yes | How long (in minutes) the bot will wait for an unfilled buy order to complete, after which the order will be cancelled.
| `unfilledtimeout.sell` | 10 | Yes | How long (in minutes) the bot will wait for an unfilled sell order to complete, after which the order will be cancelled.
| `bid_strategy.ask_last_balance` | 0.0 | Yes | Set the bidding price. More information below.

182
docs/installation.md
View File

@ -8,7 +8,6 @@ To understand how to set up the bot please read the [Bot Configuration](https://
* [Table of Contents](#table-of-contents)
* [Easy Installation - Linux Script](#easy-installation---linux-script)
* [Manual installation](#manual-installation)
* [Automatic Installation - Docker](#automatic-installation---docker)
* [Custom Linux MacOS Installation](#custom-installation)
- [Requirements](#requirements)
@ -56,34 +55,6 @@ Reset parameter will hard reset your branch (only if you are on `master` or `dev
Config parameter is a `config.json` configurator. This script will ask you questions to setup your bot and create your `config.json`.
## Manual installation - Linux/MacOS
The following steps are made for Linux/MacOS environment
### 1. Clone the repo
```bash
git clone git@github.com:freqtrade/freqtrade.git
git checkout develop
cd freqtrade
```
### 2. Create the config file
Switch `"dry_run": true,`
```bash
cp config.json.example config.json
vi config.json
```
### 3. Build your docker image and run it
```bash
docker build -t freqtrade .
docker run --rm -v /etc/localtime:/etc/localtime:ro -v `pwd`/config.json:/freqtrade/config.json -it freqtrade
```
------
## Automatic Installation - Docker
@ -196,7 +167,7 @@ docker run -d \
freqtrade --db-url sqlite:///tradesv3.sqlite
```
NOTE: db-url defaults to `sqlite:///tradesv3.sqlite` but it defaults to `sqlite://` if `dry_run=True` is being used.
*Note*: db-url defaults to `sqlite:///tradesv3.sqlite` but it defaults to `sqlite://` if `dry_run=True` is being used.
To override this behaviour use a custom db-url value: i.e.: `--db-url sqlite:///tradesv3.dryrun.sqlite`
### 6. Monitor your Docker instance
@ -211,14 +182,15 @@ docker stop freqtrade
docker start freqtrade
```
You do not need to rebuild the image for configuration changes, it will suffice to edit `config.json` and restart the container.
For more information on how to operate Docker, please refer to the [official Docker documentation](https://docs.docker.com/).
*Note*: You do not need to rebuild the image for configuration changes, it will suffice to edit `config.json` and restart the container.
### 7. Backtest with docker
The following assumes that the above steps (1-4) have been completed successfully.
Also, backtest-data should be available at `~/.freqtrade/user_data/`.
``` bash
docker run -d \
--name freqtrade \
@ -238,12 +210,13 @@ Head over to the [Backtesting Documentation](https://github.com/freqtrade/freqtr
## Custom Installation
We've included/collected install instructions for Ubuntu 16.04, MacOS, and Windows. These are guidelines and your success may vary with other distros.
OS Specific steps are listed first, the [common](#common) section below is necessary for all systems.
### Requirements
Click each one for install guide:
* [Python 3.6.x](http://docs.python-guide.org/en/latest/starting/installation/), note the bot was not tested on Python >= 3.7.x
* [Python >= 3.6.x](http://docs.python-guide.org/en/latest/starting/installation/)
* [pip](https://pip.pypa.io/en/stable/installing/)
* [git](https://git-scm.com/book/en/v2/Getting-Started-Installing-Git)
* [virtualenv](https://virtualenv.pypa.io/en/stable/installation/) (Recommended)
@ -251,7 +224,7 @@ Click each one for install guide:
### Linux - Ubuntu 16.04
#### 1. Install Python 3.6, Git, and wget
#### Install Python 3.6, Git, and wget
```bash
sudo add-apt-repository ppa:jonathonf/python-3.6
@ -259,7 +232,34 @@ sudo apt-get update
sudo apt-get install python3.6 python3.6-venv python3.6-dev build-essential autoconf libtool pkg-config make wget git
```
#### 2. Install TA-Lib
#### Raspberry Pi / Raspbian
Before installing FreqTrade on a Raspberry Pi running the official Raspbian Image, make sure you have at least Python 3.6 installed. The default image only provides Python 3.5. Probably the easiest way to get a recent version of python is [miniconda](https://repo.continuum.io/miniconda/).
The following assumes that miniconda3 is installed and available in your environment, and is installed.
It's recommended to use (mini)conda for this as installation/compilation of `scipy` and `pandas` takes a long time.
``` bash
conda config --add channels rpi
conda install python=3.6
conda create -n freqtrade python=3.6
conda install scipy pandas
pip install -r requirements.txt
pip install -e .
```
### MacOS
#### Install Python 3.6, git, wget and ta-lib
```bash
brew install python3 git wget
```
### common
#### 1. Install TA-Lib
Official webpage: https://mrjbq7.github.io/ta-lib/install.html
@ -275,15 +275,60 @@ cd ..
rm -rf ./ta-lib*
```
*Note*: An already downloaded version of ta-lib is included in the repository, as the sourceforge.net source seems to have problems frequently.
#### 2. Setup your Python virtual environment (virtualenv)
*Note*: This step is optional but strongly recommended to keep your system organized
```bash
python3 -m venv .env
source .env/bin/activate
```
#### 3. Install FreqTrade
Clone the git repository:
```bash
git clone https://github.com/freqtrade/freqtrade.git
```
#### 4. Configure `freqtrade` as a `systemd` service
Optionally checkout the stable/master branch:
```bash
git checkout master
```
#### 4. Initialize the configuration
```bash
cd freqtrade
cp config.json.example config.json
```
> *To edit the config please refer to [Bot Configuration](https://github.com/freqtrade/freqtrade/blob/develop/docs/configuration.md).*
#### 5. Install python dependencies
``` bash
pip3 install --upgrade pip
pip3 install -r requirements.txt
pip3 install -e .
```
#### 6. Run the Bot
If this is the first time you run the bot, ensure you are running it in Dry-run `"dry_run": true,` otherwise it will start to buy and sell coins.
```bash
python3.6 ./freqtrade/main.py -c config.json
```
*Note*: If you run the bot on a server, you should consider using [Docker](#automatic-installation---docker) a terminal multiplexer like `screen` or [`tmux`](https://en.wikipedia.org/wiki/Tmux) to avoid that the bot is stopped on logout.
#### 7. [Optional] Configure `freqtrade` as a `systemd` service
From the freqtrade repo... copy `freqtrade.service` to your systemd user directory (usually `~/.config/systemd/user`) and update `WorkingDirectory` and `ExecStart` to match your setup.
@ -299,57 +344,6 @@ For this to be persistent (run when user is logged out) you'll need to enable `l
sudo loginctl enable-linger "$USER"
```
### MacOS
#### 1. Install Python 3.6, git, wget and ta-lib
```bash
brew install python3 git wget ta-lib
```
#### 2. Install FreqTrade
Clone the git repository:
```bash
git clone https://github.com/freqtrade/freqtrade.git
```
Optionally checkout the develop branch:
```bash
git checkout develop
```
### Setup Config and virtual env
#### 1. Initialize the configuration
```bash
cd freqtrade
cp config.json.example config.json
```
> *To edit the config please refer to [Bot Configuration](https://github.com/freqtrade/freqtrade/blob/develop/docs/configuration.md).*
#### 2. Setup your Python virtual environment (virtualenv)
```bash
python3.6 -m venv .env
source .env/bin/activate
pip3.6 install --upgrade pip
pip3.6 install -r requirements.txt
pip3.6 install -e .
```
#### 3. Run the Bot
If this is the first time you run the bot, ensure you are running it in Dry-run `"dry_run": true,` otherwise it will start to buy and sell coins.
```bash
python3.6 ./freqtrade/main.py -c config.json
```
------
## Windows
@ -369,7 +363,7 @@ git clone https://github.com/freqtrade/freqtrade.git
copy paste `config.json` to ``\path\freqtrade-develop\freqtrade`
#### install ta-lib
#### Install ta-lib
Install ta-lib according to the [ta-lib documentation](https://github.com/mrjbq7/ta-lib#windows).
@ -390,5 +384,17 @@ REM >pip install TA_Lib‑0.4.17‑cp36‑cp36m‑win32.whl
> Thanks [Owdr](https://github.com/Owdr) for the commands. Source: [Issue #222](https://github.com/freqtrade/freqtrade/issues/222)
#### Error during installation under Windows
``` bash
error: Microsoft Visual C++ 14.0 is required. Get it with "Microsoft Visual C++ Build Tools": http://landinghub.visualstudio.com/visual-cpp-build-tools
```
Unfortunately, many packages requiring compilation don't provide a pre-build wheel. It is therefore mandatory to have a C/C++ compiler installed and available for your python environment to use.
The easiest way is to download install Microsoft Visual Studio Community [here](https://visualstudio.microsoft.com/downloads/) and make sure to install "Common Tools for Visual C++" to enable building c code on Windows. Unfortunately, this is a heavy download / dependency (~4Gb) so you might want to consider WSL or docker first.
---
Now you have an environment ready, the next step is
[Bot Configuration](https://github.com/freqtrade/freqtrade/blob/develop/docs/configuration.md)...

1
freqtrade/constants.py
View File

@ -53,6 +53,7 @@ CONF_SCHEMA = {
},
'fiat_display_currency': {'type': 'string', 'enum': SUPPORTED_FIAT},
'dry_run': {'type': 'boolean'},
'process_only_new_candles': {'type': 'boolean'},
'minimal_roi': {
'type': 'object',
'patternProperties': {

33
freqtrade/strategy/interface.py
View File

@ -70,8 +70,15 @@ class IStrategy(ABC):
# associated ticker interval
ticker_interval: str
# run "populate_indicators" only for new candle
process_only_new_candles: bool = False
# Dict to determine if analysis is necessary
_last_candle_seen_per_pair: Dict[str, datetime] = {}
def __init__(self, config: dict) -> None:
self.config = config
self._last_candle_seen_per_pair = {}
@abstractmethod
def populate_indicators(self, dataframe: DataFrame, metadata: dict) -> DataFrame:
@ -112,10 +119,30 @@ class IStrategy(ABC):
add several TA indicators and buy signal to it
:return DataFrame with ticker data and indicator data
"""
dataframe = parse_ticker_dataframe(ticker_history)
dataframe = self.advise_indicators(dataframe, metadata)
dataframe = self.advise_buy(dataframe, metadata)
dataframe = self.advise_sell(dataframe, metadata)
pair = str(metadata.get('pair'))
# Test if seen this pair and last candle before.
# always run if process_only_new_candles is set to true
if (not self.process_only_new_candles or
self._last_candle_seen_per_pair.get(pair, None) != dataframe.iloc[-1]['date']):
# Defs that only make change on new candle data.
logging.debug("TA Analysis Launched")
dataframe = self.advise_indicators(dataframe, metadata)
dataframe = self.advise_buy(dataframe, metadata)
dataframe = self.advise_sell(dataframe, metadata)
self._last_candle_seen_per_pair[pair] = dataframe.iloc[-1]['date']
else:
logging.debug("Skippinig TA Analysis for already analyzed candle")
dataframe['buy'] = 0
dataframe['sell'] = 0
# Other Defs in strategy that want to be called every loop here
# twitter_sell = self.watch_twitter_feed(dataframe, metadata)
logging.debug("Loop Analysis Launched")
return dataframe
def get_signal(self, pair: str, interval: str,

9
freqtrade/strategy/resolver.py
View File

@ -66,6 +66,15 @@ class StrategyResolver(object):
else:
config['ticker_interval'] = self.strategy.ticker_interval
if 'process_only_new_candles' in config:
self.strategy.process_only_new_candles = config['process_only_new_candles']
logger.info(
"Override process_only_new_candles 'process_only_new_candles' "
"with value in config file: %s.", config['process_only_new_candles']
)
else:
config['process_only_new_candles'] = self.strategy.process_only_new_candles
# Sort and apply type conversions
self.strategy.minimal_roi = OrderedDict(sorted(
{int(key): value for (key, value) in self.strategy.minimal_roi.items()}.items(),

72
freqtrade/tests/strategy/test_interface.py
View File

@ -128,3 +128,75 @@ def test_min_roi_reached(default_conf, fee) -> None:
assert not strategy.min_roi_reached(trade, -0.01, arrow.utcnow().shift(minutes=-1).datetime)
assert strategy.min_roi_reached(trade, 0.02, arrow.utcnow().shift(minutes=-1).datetime)
def test_analyze_ticker_default(ticker_history, mocker, caplog) -> None:
caplog.set_level(logging.DEBUG)
ind_mock = MagicMock(side_effect=lambda x, meta: x)
buy_mock = MagicMock(side_effect=lambda x, meta: x)
sell_mock = MagicMock(side_effect=lambda x, meta: x)
mocker.patch.multiple(
'freqtrade.strategy.interface.IStrategy',
advise_indicators=ind_mock,
advise_buy=buy_mock,
advise_sell=sell_mock,
)
strategy = DefaultStrategy({})
strategy.analyze_ticker(ticker_history, {'pair': 'ETH/BTC'})
assert ind_mock.call_count == 1
assert buy_mock.call_count == 1
assert buy_mock.call_count == 1
assert log_has('TA Analysis Launched', caplog.record_tuples)
assert not log_has('Skippinig TA Analysis for already analyzed candle',
caplog.record_tuples)
caplog.clear()
strategy.analyze_ticker(ticker_history, {'pair': 'ETH/BTC'})
# No analysis happens as process_only_new_candles is true
assert ind_mock.call_count == 2
assert buy_mock.call_count == 2
assert buy_mock.call_count == 2
assert log_has('TA Analysis Launched', caplog.record_tuples)
assert not log_has('Skippinig TA Analysis for already analyzed candle',
caplog.record_tuples)
def test_analyze_ticker_skip_analyze(ticker_history, mocker, caplog) -> None:
caplog.set_level(logging.DEBUG)
ind_mock = MagicMock(side_effect=lambda x, meta: x)
buy_mock = MagicMock(side_effect=lambda x, meta: x)
sell_mock = MagicMock(side_effect=lambda x, meta: x)
mocker.patch.multiple(
'freqtrade.strategy.interface.IStrategy',
advise_indicators=ind_mock,
advise_buy=buy_mock,
advise_sell=sell_mock,
)
strategy = DefaultStrategy({})
strategy.process_only_new_candles = True
ret = strategy.analyze_ticker(ticker_history, {'pair': 'ETH/BTC'})
assert ind_mock.call_count == 1
assert buy_mock.call_count == 1
assert buy_mock.call_count == 1
assert log_has('TA Analysis Launched', caplog.record_tuples)
assert not log_has('Skippinig TA Analysis for already analyzed candle',
caplog.record_tuples)
caplog.clear()
ret = strategy.analyze_ticker(ticker_history, {'pair': 'ETH/BTC'})
# No analysis happens as process_only_new_candles is true
assert ind_mock.call_count == 1
assert buy_mock.call_count == 1
assert buy_mock.call_count == 1
# only skipped analyze adds buy and sell columns, otherwise it's all mocked
assert 'buy' in ret
assert 'sell' in ret
assert ret['buy'].sum() == 0
assert ret['sell'].sum() == 0
assert not log_has('TA Analysis Launched', caplog.record_tuples)
assert log_has('Skippinig TA Analysis for already analyzed candle',
caplog.record_tuples)

17
freqtrade/tests/strategy/test_strategy.py
View File

@ -165,6 +165,23 @@ def test_strategy_override_ticker_interval(caplog):
) in caplog.record_tuples
def test_strategy_override_process_only_new_candles(caplog):
caplog.set_level(logging.INFO)
config = {
'strategy': 'DefaultStrategy',
'process_only_new_candles': True
}
resolver = StrategyResolver(config)
assert resolver.strategy.process_only_new_candles
assert ('freqtrade.strategy.resolver',
logging.INFO,
"Override process_only_new_candles 'process_only_new_candles' "
"with value in config file: True."
) in caplog.record_tuples
def test_deprecate_populate_indicators(result):
default_location = path.join(path.dirname(path.realpath(__file__)))
resolver = StrategyResolver({'strategy': 'TestStrategyLegacy',

8
requirements.txt
View File

@ -1,6 +1,6 @@
ccxt==1.17.205
ccxt==1.17.233
SQLAlchemy==1.2.11
python-telegram-bot==10.1.0
python-telegram-bot==11.1.0
arrow==0.12.1
cachetools==2.1.0
requests==2.19.1
@ -12,10 +12,10 @@ scipy==1.1.0
jsonschema==2.6.0
numpy==1.15.1
TA-Lib==0.4.17
pytest==3.7.3
pytest==3.7.4
pytest-mock==1.10.0
pytest-cov==2.5.1
pytest-asyncio==0.9.0
pytest-cov==2.6.0
tabulate==0.8.2
coinmarketcap==5.0.3

3
user_data/strategies/test_strategy.py
View File

@ -45,6 +45,9 @@ class TestStrategy(IStrategy):
# Optimal ticker interval for the strategy
ticker_interval = '5m'
# run "populate_indicators" only for new candle
ta_on_candle = False
def populate_indicators(self, dataframe: DataFrame, metadata: dict) -> DataFrame:
"""
Adds several different TA indicators to the given DataFrame